Jul 22 16:22:37.190392 ip-10-0-138-200 systemd[1]: Starting Kubernetes Kubelet... Jul 22 16:22:37.517033 ip-10-0-138-200 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:22:37.517033 ip-10-0-138-200 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 16:22:37.517033 ip-10-0-138-200 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:22:37.517033 ip-10-0-138-200 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 16:22:37.517033 ip-10-0-138-200 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:22:37.519265 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.519157 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 16:22:37.523922 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523904 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:22:37.523922 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523922 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523926 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523932 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523936 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523939 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523942 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523944 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523947 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523950 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523952 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523955 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523957 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523959 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523962 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523965 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523967 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523970 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523972 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523974 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:22:37.523988 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523978 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523982 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523985 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523988 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523991 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523993 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523996 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.523999 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524001 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524004 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524006 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524009 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524012 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524014 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524017 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524019 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524022 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524024 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524026 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524030 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:22:37.524454 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524033 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524036 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524039 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524042 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524044 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524047 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524049 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524052 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524054 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524056 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524059 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524061 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524064 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524068 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524071 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524074 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524077 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524080 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524082 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524099 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:22:37.524961 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524102 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524104 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524107 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524109 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524112 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524114 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524117 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524119 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524122 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524124 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524127 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524130 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524133 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524135 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524137 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524140 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524142 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524144 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524147 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524151 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:22:37.525465 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524153 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524156 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524158 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524161 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524163 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524166 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524170 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524173 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524175 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524178 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524180 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524183 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524186 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524188 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524190 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524193 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524196 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524198 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524201 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524904 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:22:37.525960 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524910 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524913 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524915 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524918 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524921 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524923 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524926 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524928 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524931 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524933 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524935 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524938 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524941 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524943 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524946 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524948 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524950 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524953 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524957 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:22:37.526463 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524960 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524963 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524965 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524968 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524970 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524973 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524976 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524979 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524982 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524984 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524987 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524990 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524992 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524995 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.524997 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525000 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525002 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525004 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525008 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525012 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:22:37.526924 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525015 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525017 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525020 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525022 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525025 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525027 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525030 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525032 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525034 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525037 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525040 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525043 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525045 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525048 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525051 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525053 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525056 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525059 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525061 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525064 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:22:37.527421 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525066 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525068 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525072 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525074 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525077 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525079 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525082 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525099 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525102 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525104 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525106 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525109 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525111 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525114 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525116 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525119 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525121 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525123 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525126 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:22:37.527901 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525129 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525133 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525136 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525139 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525143 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525146 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525149 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525152 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525155 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525158 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525160 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525162 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525165 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525167 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525171 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525174 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525177 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525179 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525182 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:22:37.528383 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525185 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525561 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525566 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525569 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525572 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525575 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525577 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525580 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525583 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525585 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525587 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525590 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525593 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525595 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525598 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525600 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525603 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525606 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525609 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525612 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:22:37.528857 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525615 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525617 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525620 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525624 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525627 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525630 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525632 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525635 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525638 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525640 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525643 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525645 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525648 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525650 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525653 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525655 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525657 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525660 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525662 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:22:37.529341 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525665 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525667 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525670 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525673 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525676 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525678 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525681 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525683 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525686 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525688 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525692 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525695 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525697 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525700 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525703 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525705 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525708 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525710 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525712 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525715 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:22:37.529819 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525717 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525720 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525723 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525725 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525728 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525730 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525732 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525735 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525737 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525740 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525742 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525745 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525747 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525750 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525752 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525755 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525759 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525762 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525764 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525767 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:22:37.530362 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525770 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525773 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525776 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525778 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525780 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525783 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525785 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525788 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525791 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525793 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525796 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525799 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525801 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525803 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525806 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525808 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525812 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525815 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525818 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:22:37.530841 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525821 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.525824 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525900 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525906 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525915 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525919 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525923 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525926 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525930 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525934 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525937 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525940 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525944 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525947 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525949 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525953 2555 flags.go:64] FLAG: --cgroup-root="" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525955 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525958 2555 flags.go:64] FLAG: --client-ca-file="" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525961 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525964 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525969 2555 flags.go:64] FLAG: --cluster-domain="" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525972 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525976 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525979 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 16:22:37.531309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525982 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525985 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525988 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525991 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525994 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.525997 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526000 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526002 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526006 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526009 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526013 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526015 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526021 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526024 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526027 2555 flags.go:64] FLAG: --enable-server="true" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526030 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526034 2555 flags.go:64] FLAG: --event-burst="100" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526037 2555 flags.go:64] FLAG: --event-qps="50" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526040 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526043 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526046 2555 flags.go:64] FLAG: --eviction-hard="" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526049 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526052 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526059 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526061 2555 flags.go:64] FLAG: --eviction-soft="" Jul 22 16:22:37.531898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526064 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526067 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526070 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526073 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526075 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526079 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526082 2555 flags.go:64] FLAG: --feature-gates="" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526101 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526104 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526107 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526110 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526113 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526116 2555 flags.go:64] FLAG: --help="false" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526119 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526122 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526125 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526127 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526131 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526134 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526138 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526141 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526143 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526146 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 16:22:37.532511 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526149 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526152 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526155 2555 flags.go:64] FLAG: --kube-reserved="" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526157 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526160 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526163 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526166 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526170 2555 flags.go:64] FLAG: --lock-file="" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526172 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526175 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526178 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526183 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526185 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526188 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526191 2555 flags.go:64] FLAG: --logging-format="text" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526194 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526197 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526200 2555 flags.go:64] FLAG: --manifest-url="" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526203 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526207 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526209 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526213 2555 flags.go:64] FLAG: --max-pods="110" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526216 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526219 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526221 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 16:22:37.533268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526225 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526228 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526230 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526235 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526246 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526249 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526252 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526255 2555 flags.go:64] FLAG: --pod-cidr="" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526258 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526264 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526266 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526269 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526272 2555 flags.go:64] FLAG: --port="10250" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526275 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526278 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-050dd7a4c3de7f897" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526281 2555 flags.go:64] FLAG: --qos-reserved="" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526285 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526288 2555 flags.go:64] FLAG: --register-node="true" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526291 2555 flags.go:64] FLAG: --register-with-taints="" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526294 2555 flags.go:64] FLAG: --registry-burst="10" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526297 2555 flags.go:64] FLAG: --registry-qps="5" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526300 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526302 2555 flags.go:64] FLAG: --reserved-memory="" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526306 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526310 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 16:22:37.534200 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526313 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526315 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526318 2555 flags.go:64] FLAG: --runonce="false" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526321 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526324 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526326 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526329 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526332 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526334 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526338 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526340 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526345 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526348 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526351 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526353 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526356 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526359 2555 flags.go:64] FLAG: --system-cgroups="" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526362 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526367 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526369 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526372 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526377 2555 flags.go:64] FLAG: --tls-min-version="" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526380 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526383 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526385 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 16:22:37.534812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526388 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526391 2555 flags.go:64] FLAG: --v="2" Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526395 2555 flags.go:64] FLAG: --version="false" Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526399 2555 flags.go:64] FLAG: --vmodule="" Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526402 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.526406 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526518 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526521 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526526 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526529 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526532 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526535 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526537 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526540 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526543 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526546 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526549 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526551 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526555 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526557 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526560 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:22:37.535604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526563 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526565 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526568 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526570 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526573 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526575 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526578 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526581 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526584 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526586 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526589 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526591 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526594 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526596 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526599 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526601 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526604 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526606 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526609 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526612 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:22:37.536134 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526614 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526616 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526619 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526621 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526623 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526626 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526628 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526631 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526633 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526637 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526639 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526641 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526644 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526646 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526649 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526652 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526655 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526658 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526660 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526667 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:22:37.536625 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526669 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526672 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526674 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526677 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526680 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526682 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526685 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526687 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526689 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526692 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526694 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526697 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526700 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526704 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526707 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526710 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526712 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526715 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526718 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:22:37.537115 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526721 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526723 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526728 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526730 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526733 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526735 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526738 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526740 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526743 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526745 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526747 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526750 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526754 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526756 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526759 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526761 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526763 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526766 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526768 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526771 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:22:37.537590 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526773 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526776 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526778 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526780 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.526783 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.527442 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.535310 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.535326 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.535346 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.535352 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535409 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535414 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535418 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535420 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:22:37.538073 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535424 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535426 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535429 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535432 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535435 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535437 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535440 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535442 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535445 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535447 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535451 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535454 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535456 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535458 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535461 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535465 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535469 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535472 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535474 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:22:37.538456 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535477 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535480 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535482 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535485 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535488 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535492 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535495 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535498 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535501 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535504 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535507 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535509 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535512 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535515 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535517 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535520 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535522 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535524 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535527 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:22:37.538970 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535529 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535532 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535534 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535536 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535540 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535542 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535545 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535547 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535549 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535552 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535554 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535557 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535559 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535562 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535564 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535567 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535569 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535572 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535575 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535578 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:22:37.539608 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535581 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535583 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535586 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535588 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535592 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535595 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535598 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535600 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535603 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535605 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535608 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535611 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535613 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535617 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535621 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535624 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535627 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535630 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535633 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:22:37.540107 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535636 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535638 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535641 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535643 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535646 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535649 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535651 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535653 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535656 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535659 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535661 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535663 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535666 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535668 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535670 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535673 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535675 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:22:37.540578 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535677 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.535683 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535792 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535797 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535801 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535803 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535806 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535808 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535811 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535813 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535816 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535818 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535820 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535823 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535826 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:22:37.540980 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535829 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535831 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535833 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535836 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535839 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535842 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535844 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535847 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535849 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535852 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535855 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535858 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535860 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535863 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535865 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535868 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535871 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535874 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535876 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535879 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:22:37.541353 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535881 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535884 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535886 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535888 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535891 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535893 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535896 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535898 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535900 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535903 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535905 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535907 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535912 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535914 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535916 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535919 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535921 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535924 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535926 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535929 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:22:37.541829 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535932 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535934 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535937 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535939 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535942 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535945 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535947 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535949 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535952 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535954 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535958 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535961 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535964 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535966 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535969 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535972 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535974 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535976 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535979 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:22:37.542351 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535982 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535985 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535987 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535990 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535993 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.535995 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536000 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536003 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536006 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536009 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536012 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536015 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536018 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536020 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536023 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536025 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536028 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536030 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536032 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:22:37.542798 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536035 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536037 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536039 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536042 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536044 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536046 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536049 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:37.536051 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.536056 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.536180 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.538219 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.539352 2555 server.go:1019] "Starting client certificate rotation" Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.539448 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:22:37.543308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.539476 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:22:37.543621 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.542030 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:22:37.543621 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.543501 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:22:37.554824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.554804 2555 log.go:25] "Validated CRI v1 runtime API" Jul 22 16:22:37.556947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.556932 2555 log.go:25] "Validated CRI v1 image API" Jul 22 16:22:37.558809 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.558794 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 16:22:37.560966 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.560948 2555 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b87c59fe-2a90-440b-bd66-f28a294677e0:/dev/nvme0n1p3 c6f78c1a-3ef9-4aa6-9ab2-1fa99174b3c7:/dev/nvme0n1p4] Jul 22 16:22:37.561023 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.560972 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 16:22:37.566020 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.565893 2555 manager.go:216] Machine: {Timestamp:2026-07-22 16:22:37.564674773 +0000 UTC m=+0.270117306 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101990 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec244e71b52e95f426df5afe7ffd5dad SystemUUID:ec244e71-b52e-95f4-26df-5afe7ffd5dad BootID:000c6a55-152a-4765-9408-bfb14a2e4a24 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ae:a2:45:bb:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ae:a2:45:bb:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:ca:cf:69:e9:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 16:22:37.566020 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.566012 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 16:22:37.566193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.566105 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 16:22:37.568648 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.568622 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 16:22:37.568839 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.568652 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-200.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 16:22:37.568912 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.568874 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:22:37.569397 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.569385 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 16:22:37.569446 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.569402 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 16:22:37.569446 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.569414 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:22:37.569446 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.569439 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:22:37.570711 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.570699 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 16:22:37.574554 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.574540 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:22:37.574689 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.574678 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 16:22:37.574744 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.574715 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 22 16:22:37.574744 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.574735 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 16:22:37.574841 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.574754 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 16:22:37.574841 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.574767 2555 kubelet.go:398] "Adding apiserver pod source" Jul 22 16:22:37.574841 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.574780 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 16:22:37.576275 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.576241 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:22:37.576356 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.576285 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:22:37.579375 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.579354 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 16:22:37.581177 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.581163 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 16:22:37.581227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.581194 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 16:22:37.582591 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582580 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582596 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582603 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582609 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582618 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582623 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582631 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582636 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582643 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582649 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582657 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 16:22:37.582674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.582663 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 16:22:37.583262 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.583252 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 16:22:37.583262 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.583261 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 16:22:37.585542 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.585521 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-200.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 16:22:37.585622 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.585582 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 16:22:37.585622 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.585577 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-200.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 16:22:37.586561 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.586548 2555 server.go:1268] "Started kubelet" Jul 22 16:22:37.586638 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.586611 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 16:22:37.586712 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.586659 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 16:22:37.586772 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.586760 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 16:22:37.587388 ip-10-0-138-200 systemd[1]: Started Kubernetes Kubelet. Jul 22 16:22:37.588269 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.588208 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 16:22:37.588438 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.588426 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 22 16:22:37.588523 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.588506 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9785t" Jul 22 16:22:37.591406 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.590668 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-200.ec2.internal.18c4a92bb8505bb6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-200.ec2.internal,UID:ip-10-0-138-200.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-200.ec2.internal,},FirstTimestamp:2026-07-22 16:22:37.586521014 +0000 UTC m=+0.291963604,LastTimestamp:2026-07-22 16:22:37.586521014 +0000 UTC m=+0.291963604,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-200.ec2.internal,}" Jul 22 16:22:37.592351 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.592326 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 16:22:37.594616 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.594602 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 16:22:37.594688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.594607 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 16:22:37.594737 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.594698 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:37.595145 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595129 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 16:22:37.595235 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595148 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 16:22:37.595235 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595131 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 16:22:37.595345 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595267 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 22 16:22:37.595345 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595275 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 22 16:22:37.595504 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595485 2555 factory.go:55] Registering systemd factory Jul 22 16:22:37.595504 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595507 2555 factory.go:223] Registration of the systemd container factory successfully Jul 22 16:22:37.595816 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595762 2555 factory.go:153] Registering CRI-O factory Jul 22 16:22:37.595816 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595776 2555 factory.go:223] Registration of the crio container factory successfully Jul 22 16:22:37.595913 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595830 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 16:22:37.595913 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595855 2555 factory.go:103] Registering Raw factory Jul 22 16:22:37.595913 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.595869 2555 manager.go:1196] Started watching for new ooms in manager Jul 22 16:22:37.596448 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.596424 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9785t" Jul 22 16:22:37.596558 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.596546 2555 manager.go:349] Starting recovery of all containers Jul 22 16:22:37.597425 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.597401 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 16:22:37.597543 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.597521 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-200.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 16:22:37.607195 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.607177 2555 manager.go:354] Recovery completed Jul 22 16:22:37.611518 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.611415 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:22:37.613260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.613171 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:22:37.613260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.613201 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:22:37.613260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.613215 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:22:37.613730 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.613714 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 16:22:37.613730 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.613728 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 16:22:37.613860 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.613729 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:22:37.613860 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.613743 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:22:37.614429 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.614413 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:22:37.614507 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.614445 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:22:37.614507 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.614460 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:22:37.615863 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.615850 2555 policy_none.go:49] "None policy: Start" Jul 22 16:22:37.615924 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.615867 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 16:22:37.615924 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.615878 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 16:22:37.617888 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.617875 2555 policy_none.go:47] "Start" Jul 22 16:22:37.663668 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.663644 2555 manager.go:337] "Starting Device Plugin manager" Jul 22 16:22:37.663766 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.663677 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 16:22:37.663766 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.663688 2555 server.go:87] "Starting device plugin registration server" Jul 22 16:22:37.663928 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.663916 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 16:22:37.663967 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.663932 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 16:22:37.664046 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.664031 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 16:22:37.664147 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.664136 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 16:22:37.664207 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.664148 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 16:22:37.664653 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.664632 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 16:22:37.664753 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.664689 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:37.736665 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.736626 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 16:22:37.738853 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.738021 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 16:22:37.738853 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.738061 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 16:22:37.738853 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.738108 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 16:22:37.738853 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.738145 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 16:22:37.740528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.740505 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:22:37.764578 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.764557 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:22:37.765595 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.765574 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:22:37.765688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.765608 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:22:37.765688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.765622 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:22:37.765688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.765652 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.774109 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.774048 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.774109 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.774071 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-200.ec2.internal\": node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:37.795806 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.795788 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:37.838802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.838759 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal"] Jul 22 16:22:37.838937 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.838844 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:22:37.840320 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.840305 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:22:37.840378 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.840336 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:22:37.840378 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.840349 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:22:37.842623 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.842610 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:22:37.842747 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.842725 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.842840 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.842757 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:22:37.843223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.843207 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:22:37.843314 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.843230 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:22:37.843314 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.843244 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:22:37.843418 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.843313 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:22:37.843418 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.843339 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:22:37.843418 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.843354 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:22:37.845592 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.845576 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.845675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.845606 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:22:37.846235 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.846219 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:22:37.846322 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.846250 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:22:37.846322 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.846262 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:22:37.865773 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.865754 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-200.ec2.internal\" not found" node="ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.870284 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.870268 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-200.ec2.internal\" not found" node="ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.896044 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.896012 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:37.897584 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.897565 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec0a202b202e3a58b24c675cdca1845c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal\" (UID: \"ec0a202b202e3a58b24c675cdca1845c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.897662 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.897596 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec0a202b202e3a58b24c675cdca1845c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal\" (UID: \"ec0a202b202e3a58b24c675cdca1845c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.897662 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.897621 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7300a218e12596f33fabcbb2467c251-config\") pod \"kube-apiserver-proxy-ip-10-0-138-200.ec2.internal\" (UID: \"a7300a218e12596f33fabcbb2467c251\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.996777 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:37.996743 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:37.997875 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.997856 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec0a202b202e3a58b24c675cdca1845c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal\" (UID: \"ec0a202b202e3a58b24c675cdca1845c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.997930 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.997887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec0a202b202e3a58b24c675cdca1845c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal\" (UID: \"ec0a202b202e3a58b24c675cdca1845c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.997930 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.997904 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7300a218e12596f33fabcbb2467c251-config\") pod \"kube-apiserver-proxy-ip-10-0-138-200.ec2.internal\" (UID: \"a7300a218e12596f33fabcbb2467c251\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.998012 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.997950 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec0a202b202e3a58b24c675cdca1845c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal\" (UID: \"ec0a202b202e3a58b24c675cdca1845c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.998012 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.997974 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7300a218e12596f33fabcbb2467c251-config\") pod \"kube-apiserver-proxy-ip-10-0-138-200.ec2.internal\" (UID: \"a7300a218e12596f33fabcbb2467c251\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" Jul 22 16:22:37.998012 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:37.997954 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec0a202b202e3a58b24c675cdca1845c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal\" (UID: \"ec0a202b202e3a58b24c675cdca1845c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:38.097522 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:38.097482 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:38.168777 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.168753 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:38.172527 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.172505 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" Jul 22 16:22:38.198418 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:38.198389 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:38.298881 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:38.298840 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:38.399424 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:38.399355 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:38.499833 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:38.499801 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:38.539058 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.539034 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 16:22:38.539544 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.539193 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 16:22:38.600330 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:38.600298 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:38.602063 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.602023 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 16:17:37 +0000 UTC" deadline="2028-05-09 13:17:29.33671592 +0000 UTC" Jul 22 16:22:38.602244 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.602063 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15764h54m50.734656999s" Jul 22 16:22:38.684677 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:38.684647 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7300a218e12596f33fabcbb2467c251.slice/crio-ea569a6882bc746b45a9e7eae24b372573c3900f7c6fcaa6cf03cc84f082f1cd WatchSource:0}: Error finding container ea569a6882bc746b45a9e7eae24b372573c3900f7c6fcaa6cf03cc84f082f1cd: Status 404 returned error can't find the container with id ea569a6882bc746b45a9e7eae24b372573c3900f7c6fcaa6cf03cc84f082f1cd Jul 22 16:22:38.685047 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:38.685033 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0a202b202e3a58b24c675cdca1845c.slice/crio-ea44f8e682f56aaff43e319c141a20b2fb9a09c5ee0809a87b9d9251083b6b06 WatchSource:0}: Error finding container ea44f8e682f56aaff43e319c141a20b2fb9a09c5ee0809a87b9d9251083b6b06: Status 404 returned error can't find the container with id ea44f8e682f56aaff43e319c141a20b2fb9a09c5ee0809a87b9d9251083b6b06 Jul 22 16:22:38.690982 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.690959 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:22:38.700504 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:38.700484 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-200.ec2.internal\" not found" Jul 22 16:22:38.741993 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.741940 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" event={"ID":"a7300a218e12596f33fabcbb2467c251","Type":"ContainerStarted","Data":"ea569a6882bc746b45a9e7eae24b372573c3900f7c6fcaa6cf03cc84f082f1cd"} Jul 22 16:22:38.742900 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.742880 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" event={"ID":"ec0a202b202e3a58b24c675cdca1845c","Type":"ContainerStarted","Data":"ea44f8e682f56aaff43e319c141a20b2fb9a09c5ee0809a87b9d9251083b6b06"} Jul 22 16:22:38.748794 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.748776 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:22:38.766842 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.766812 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:22:38.795241 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.795217 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" Jul 22 16:22:38.808228 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.808204 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:22:38.809151 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.809118 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" Jul 22 16:22:38.817072 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.817052 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:22:38.933482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:38.933397 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:22:39.576670 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.576634 2555 apiserver.go:52] "Watching apiserver" Jul 22 16:22:39.586430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.586399 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 16:22:39.586924 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.586898 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-dczqj","openshift-network-operator/iptables-alerter-nxhss","kube-system/global-pull-secret-syncer-ckfjr","kube-system/konnectivity-agent-49m6s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4","openshift-cluster-node-tuning-operator/tuned-hbpv5","openshift-multus/multus-99zkh","openshift-ovn-kubernetes/ovnkube-node-dbgk4","kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal","openshift-dns/node-resolver-szwzs","openshift-image-registry/node-ca-2pvvg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal","openshift-multus/multus-additional-cni-plugins-n2wx6","openshift-multus/network-metrics-daemon-rw9fk"] Jul 22 16:22:39.589951 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.589926 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.593525 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.593217 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.593525 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.593515 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 16:22:39.593828 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.593806 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 16:22:39.593924 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.593845 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 16:22:39.594066 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.594049 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 16:22:39.594261 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.594237 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4tfrq\"" Jul 22 16:22:39.595491 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.595471 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 16:22:39.596308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.596285 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hfb89\"" Jul 22 16:22:39.596551 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.596531 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 16:22:39.596731 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.596716 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 16:22:39.596866 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.596853 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.596960 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.596934 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:39.597997 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.597980 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.601115 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.600615 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 16:22:39.601115 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.600713 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2bpg7\"" Jul 22 16:22:39.601115 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.600805 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 16:22:39.601115 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.601004 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.603186 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.603165 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.603730 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.603704 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-42vjp\"" Jul 22 16:22:39.603730 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.603729 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 16:22:39.603892 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.603790 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 16:22:39.603892 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.603796 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 16:22:39.605523 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.605461 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:22:39.605615 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.605569 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 16:22:39.605808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.605789 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7gqbl\"" Jul 22 16:22:39.605892 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.605853 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:39.605952 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.605916 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:39.606210 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.606030 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-cni-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.606349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.606316 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29bfdb7e-ccfa-4f28-9958-fd4281974e45-cni-binary-copy\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.606414 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.606377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-system-cni-dir\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.606471 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.606417 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-daemon-config\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.606471 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.606451 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j6vt\" (UniqueName: \"kubernetes.io/projected/f7b166af-2797-4f86-b009-667c2d19b6b7-kube-api-access-5j6vt\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.606561 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.606484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df349bb5-3a16-47f3-80dc-4d13678538c2-kubelet-config\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.609227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609200 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12c56210-7d70-45ed-bb55-7ca81950c8f4-agent-certs\") pod \"konnectivity-agent-49m6s\" (UID: \"12c56210-7d70-45ed-bb55-7ca81950c8f4\") " pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.609328 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609259 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-os-release\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609328 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609293 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-netns\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609438 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-cni-multus\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609438 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609363 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-multus-certs\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609438 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609401 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.609579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609443 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-socket-dir-parent\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609504 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-cni-bin\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609677 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609639 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:22:39.609677 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609647 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.609776 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609689 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.609776 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609720 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12c56210-7d70-45ed-bb55-7ca81950c8f4-konnectivity-ca\") pod \"konnectivity-agent-49m6s\" (UID: \"12c56210-7d70-45ed-bb55-7ca81950c8f4\") " pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.609875 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609773 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-etc-kubernetes\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609875 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609812 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-system-cni-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.609875 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609859 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-726cd\" (UniqueName: \"kubernetes.io/projected/29bfdb7e-ccfa-4f28-9958-fd4281974e45-kube-api-access-726cd\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.610027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609891 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-os-release\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.610027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609925 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.610027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.609977 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-k8s-cni-cncf-io\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.610027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610019 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-kubelet\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.610255 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610060 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-conf-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.610255 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-cnibin\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.610255 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610176 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-hostroot\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.610255 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610222 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-cnibin\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.610255 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610248 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df349bb5-3a16-47f3-80dc-4d13678538c2-dbus\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.610493 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.610640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.610605 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:39.610708 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.610690 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:39.612484 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.612463 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.615506 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.615466 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.615601 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.615538 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 16:22:39.615822 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.615797 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k6vj2\"" Jul 22 16:22:39.615914 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.615840 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 16:22:39.616094 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.616074 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:22:39.618023 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.618005 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tdp5t\"" Jul 22 16:22:39.618232 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.618215 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 16:22:39.618515 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.618427 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 16:22:39.618581 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.618528 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.621247 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.621225 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.621664 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.621641 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 16:22:39.622083 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.622065 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 16:22:39.622229 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.622210 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wb5db\"" Jul 22 16:22:39.622390 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.622375 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 16:22:39.625156 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.625137 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 16:22:39.625289 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.625236 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 16:22:39.626361 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.626339 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 16:22:39.626618 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.626597 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 16:22:39.626711 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.626674 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4zqt5\"" Jul 22 16:22:39.626869 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.626849 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 16:22:39.626869 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.626605 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 16:22:39.630313 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.630256 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-27nw6" Jul 22 16:22:39.638243 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.638221 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-27nw6" Jul 22 16:22:39.697277 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.697241 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710528 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-etc-kubernetes\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710584 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-run\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710618 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l299l\" (UniqueName: \"kubernetes.io/projected/1221eced-0d60-4b12-8249-dbe37c4ba00b-kube-api-access-l299l\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710646 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-systemd\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710650 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-etc-kubernetes\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710674 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-system-cni-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710692 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-726cd\" (UniqueName: \"kubernetes.io/projected/29bfdb7e-ccfa-4f28-9958-fd4281974e45-kube-api-access-726cd\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710771 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-system-cni-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710774 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb2rk\" (UniqueName: \"kubernetes.io/projected/9090613d-21a5-4fd0-a284-6503d8fe777e-kube-api-access-kb2rk\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.710912 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711075 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-cni-netd\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711137 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-device-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711167 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711195 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-kubelet\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-conf-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.711741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711244 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-lib-modules\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711266 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-kubelet\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711290 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-var-lib-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711322 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovn-node-metrics-cert\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711331 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-kubelet\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711349 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snwsw\" (UniqueName: \"kubernetes.io/projected/a8443c5d-e428-4c06-8e61-2e442b129608-kube-api-access-snwsw\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-cnibin\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711455 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-hostroot\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711482 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-cnibin\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711507 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df349bb5-3a16-47f3-80dc-4d13678538c2-dbus\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711533 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-conf-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711534 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-systemd\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711560 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn62r\" (UniqueName: \"kubernetes.io/projected/659b0b50-5f86-4f20-a8b1-88651c23e4a2-kube-api-access-bn62r\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711590 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-cnibin\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711590 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711598 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-hostroot\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711648 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-cnibin\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.712704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711670 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1221eced-0d60-4b12-8249-dbe37c4ba00b-iptables-alerter-script\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711707 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df349bb5-3a16-47f3-80dc-4d13678538c2-dbus\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9090613d-21a5-4fd0-a284-6503d8fe777e-tmp-dir\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711743 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29bfdb7e-ccfa-4f28-9958-fd4281974e45-cni-binary-copy\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711782 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysconfig\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711806 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-kubernetes\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711871 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711908 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-systemd-units\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.711937 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-slash\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-var-lib-kubelet\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712075 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9090613d-21a5-4fd0-a284-6503d8fe777e-hosts-file\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712129 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhlgs\" (UniqueName: \"kubernetes.io/projected/abe4d75b-3d34-4b30-aabe-7952f9debdb4-kube-api-access-jhlgs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712170 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-cni-multus\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-multus-certs\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712266 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-multus-certs\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712303 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-cni-multus\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712306 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrhxw\" (UniqueName: \"kubernetes.io/projected/7e4879e5-9f03-4efa-8875-01f04f0efe05-kube-api-access-jrhxw\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.713468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712343 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-cni-bin\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.714325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712369 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29bfdb7e-ccfa-4f28-9958-fd4281974e45-cni-binary-copy\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.714325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.712368 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.714325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714051 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-env-overrides\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.714325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714141 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.714325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-socket-dir-parent\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.714325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714264 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-cni-bin\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.714325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714320 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.714704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714376 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.714704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12c56210-7d70-45ed-bb55-7ca81950c8f4-konnectivity-ca\") pod \"konnectivity-agent-49m6s\" (UID: \"12c56210-7d70-45ed-bb55-7ca81950c8f4\") " pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.714704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714434 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-etc-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.714704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714474 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-socket-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.714704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714622 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysctl-d\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.714704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714658 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovnkube-config\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.714704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714691 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-os-release\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.715051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714738 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysctl-conf\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.715051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714765 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-host\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.715051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714805 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8dw2\" (UniqueName: \"kubernetes.io/projected/e5453eb8-3321-4326-9448-debb417eddc8-kube-api-access-k8dw2\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.715051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714857 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-k8s-cni-cncf-io\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.715051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714895 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.715051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.714987 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.715051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715021 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-sys\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715056 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1221eced-0d60-4b12-8249-dbe37c4ba00b-host-slash\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715150 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e4879e5-9f03-4efa-8875-01f04f0efe05-serviceca\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715181 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-cni-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-system-cni-dir\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715272 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-modprobe-d\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715300 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5453eb8-3321-4326-9448-debb417eddc8-tmp\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-ovn\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715369 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-daemon-config\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.715398 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715398 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j6vt\" (UniqueName: \"kubernetes.io/projected/f7b166af-2797-4f86-b009-667c2d19b6b7-kube-api-access-5j6vt\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.715867 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715430 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df349bb5-3a16-47f3-80dc-4d13678538c2-kubelet-config\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.715867 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715459 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12c56210-7d70-45ed-bb55-7ca81950c8f4-agent-certs\") pod \"konnectivity-agent-49m6s\" (UID: \"12c56210-7d70-45ed-bb55-7ca81950c8f4\") " pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.715867 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e5453eb8-3321-4326-9448-debb417eddc8-etc-tuned\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.716012 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715962 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e4879e5-9f03-4efa-8875-01f04f0efe05-host\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.716012 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.715999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-node-log\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.716112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716034 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-os-release\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.716112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-netns\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.716112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716093 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.716295 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716124 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-registration-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.716295 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716187 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-run-netns\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.716295 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716210 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-log-socket\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.716295 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716249 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovnkube-script-lib\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.716295 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716278 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-sys-fs\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.716832 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716810 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-cni-binary-copy\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.716910 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716885 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-socket-dir-parent\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.716969 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.716930 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-var-lib-cni-bin\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.717067 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.717050 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.717684 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.717663 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-os-release\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.717784 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.717722 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-k8s-cni-cncf-io\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.717853 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.717810 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:39.717908 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.717895 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret podName:df349bb5-3a16-47f3-80dc-4d13678538c2 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:40.217872965 +0000 UTC m=+2.923315502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret") pod "global-pull-secret-syncer-ckfjr" (UID: "df349bb5-3a16-47f3-80dc-4d13678538c2") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:39.718808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718143 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df349bb5-3a16-47f3-80dc-4d13678538c2-kubelet-config\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:39.718808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718171 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12c56210-7d70-45ed-bb55-7ca81950c8f4-konnectivity-ca\") pod \"konnectivity-agent-49m6s\" (UID: \"12c56210-7d70-45ed-bb55-7ca81950c8f4\") " pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.718808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718240 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7b166af-2797-4f86-b009-667c2d19b6b7-system-cni-dir\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.718808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718325 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-cni-dir\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.718808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718395 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-os-release\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.718808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29bfdb7e-ccfa-4f28-9958-fd4281974e45-host-run-netns\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.718808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718669 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 16:22:39.719237 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.718895 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7b166af-2797-4f86-b009-667c2d19b6b7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.719515 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.719492 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29bfdb7e-ccfa-4f28-9958-fd4281974e45-multus-daemon-config\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.722680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.722662 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12c56210-7d70-45ed-bb55-7ca81950c8f4-agent-certs\") pod \"konnectivity-agent-49m6s\" (UID: \"12c56210-7d70-45ed-bb55-7ca81950c8f4\") " pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.723172 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.723094 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-726cd\" (UniqueName: \"kubernetes.io/projected/29bfdb7e-ccfa-4f28-9958-fd4281974e45-kube-api-access-726cd\") pod \"multus-99zkh\" (UID: \"29bfdb7e-ccfa-4f28-9958-fd4281974e45\") " pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.728269 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.728243 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j6vt\" (UniqueName: \"kubernetes.io/projected/f7b166af-2797-4f86-b009-667c2d19b6b7-kube-api-access-5j6vt\") pod \"multus-additional-cni-plugins-n2wx6\" (UID: \"f7b166af-2797-4f86-b009-667c2d19b6b7\") " pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.816978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.816943 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snwsw\" (UniqueName: \"kubernetes.io/projected/a8443c5d-e428-4c06-8e61-2e442b129608-kube-api-access-snwsw\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.817202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.816986 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-systemd\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817016 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn62r\" (UniqueName: \"kubernetes.io/projected/659b0b50-5f86-4f20-a8b1-88651c23e4a2-kube-api-access-bn62r\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817092 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-systemd\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817158 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1221eced-0d60-4b12-8249-dbe37c4ba00b-iptables-alerter-script\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817221 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9090613d-21a5-4fd0-a284-6503d8fe777e-tmp-dir\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817249 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysconfig\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817275 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-kubernetes\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817303 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817327 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysconfig\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817375 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-systemd-units\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-systemd-units\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817414 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-slash\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817443 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-var-lib-kubelet\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817468 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9090613d-21a5-4fd0-a284-6503d8fe777e-hosts-file\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817494 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhlgs\" (UniqueName: \"kubernetes.io/projected/abe4d75b-3d34-4b30-aabe-7952f9debdb4-kube-api-access-jhlgs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817505 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-slash\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817386 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-kubernetes\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrhxw\" (UniqueName: \"kubernetes.io/projected/7e4879e5-9f03-4efa-8875-01f04f0efe05-kube-api-access-jrhxw\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817568 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-cni-bin\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817595 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817613 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-env-overrides\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817613 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9090613d-21a5-4fd0-a284-6503d8fe777e-hosts-file\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817643 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817625 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9090613d-21a5-4fd0-a284-6503d8fe777e-tmp-dir\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817667 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-etc-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817683 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-socket-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817701 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysctl-d\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817717 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovnkube-config\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817735 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysctl-conf\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.817906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817750 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-host\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817764 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8dw2\" (UniqueName: \"kubernetes.io/projected/e5453eb8-3321-4326-9448-debb417eddc8-kube-api-access-k8dw2\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817765 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-etc-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817805 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-sys\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817820 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1221eced-0d60-4b12-8249-dbe37c4ba00b-host-slash\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817842 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-socket-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817851 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817875 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e4879e5-9f03-4efa-8875-01f04f0efe05-serviceca\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817896 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817899 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-modprobe-d\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817928 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5453eb8-3321-4326-9448-debb417eddc8-tmp\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817951 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-ovn\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817963 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1221eced-0d60-4b12-8249-dbe37c4ba00b-host-slash\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817979 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e5453eb8-3321-4326-9448-debb417eddc8-etc-tuned\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818013 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818020 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-env-overrides\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818028 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e4879e5-9f03-4efa-8875-01f04f0efe05-host\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.818692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818060 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-node-log\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818066 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-sys\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818071 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e4879e5-9f03-4efa-8875-01f04f0efe05-host\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818068 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1221eced-0d60-4b12-8249-dbe37c4ba00b-iptables-alerter-script\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818081 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-registration-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.818148 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.817683 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-var-lib-kubelet\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818211 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-modprobe-d\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818211 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818239 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-node-log\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818243 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-host\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818270 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-run-netns\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818296 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-log-socket\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818313 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysctl-d\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818326 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovnkube-script-lib\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818354 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-sys-fs\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818356 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-cni-bin\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.819614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818368 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-etc-sysctl-conf\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.818443 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs podName:abe4d75b-3d34-4b30-aabe-7952f9debdb4 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:40.318423196 +0000 UTC m=+3.023865719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs") pod "network-metrics-daemon-rw9fk" (UID: "abe4d75b-3d34-4b30-aabe-7952f9debdb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818488 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-sys-fs\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818546 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-run-netns\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818550 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovnkube-config\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818586 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-log-socket\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818597 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-ovn\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818619 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-run\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818627 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-registration-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818638 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l299l\" (UniqueName: \"kubernetes.io/projected/1221eced-0d60-4b12-8249-dbe37c4ba00b-kube-api-access-l299l\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818662 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-systemd\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818670 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-run\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818682 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb2rk\" (UniqueName: \"kubernetes.io/projected/9090613d-21a5-4fd0-a284-6503d8fe777e-kube-api-access-kb2rk\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818698 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818704 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-run-systemd\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-cni-netd\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818733 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-device-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.820349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818756 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818773 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-lib-modules\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-kubelet\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818824 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-cni-netd\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818835 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e4879e5-9f03-4efa-8875-01f04f0efe05-serviceca\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818842 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-var-lib-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818880 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-var-lib-openvswitch\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818886 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovn-node-metrics-cert\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818893 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818919 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovnkube-script-lib\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818925 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818966 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/659b0b50-5f86-4f20-a8b1-88651c23e4a2-host-kubelet\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818970 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5453eb8-3321-4326-9448-debb417eddc8-lib-modules\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.820960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.818967 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8443c5d-e428-4c06-8e61-2e442b129608-device-dir\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.827333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.821971 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5453eb8-3321-4326-9448-debb417eddc8-tmp\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.827333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.822098 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/659b0b50-5f86-4f20-a8b1-88651c23e4a2-ovn-node-metrics-cert\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.827333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.823555 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e5453eb8-3321-4326-9448-debb417eddc8-etc-tuned\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.827333 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.823976 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:22:39.827333 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.823997 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:22:39.827333 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.824011 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jjfxx for pod openshift-network-diagnostics/network-check-target-dczqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:39.827333 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:39.824127 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx podName:507c1878-0b1f-4510-acbf-f5ddfae2ca5c nodeName:}" failed. No retries permitted until 2026-07-22 16:22:40.324107501 +0000 UTC m=+3.029550079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jjfxx" (UniqueName: "kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx") pod "network-check-target-dczqj" (UID: "507c1878-0b1f-4510-acbf-f5ddfae2ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:39.829967 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.829751 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l299l\" (UniqueName: \"kubernetes.io/projected/1221eced-0d60-4b12-8249-dbe37c4ba00b-kube-api-access-l299l\") pod \"iptables-alerter-nxhss\" (UID: \"1221eced-0d60-4b12-8249-dbe37c4ba00b\") " pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.829967 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.829786 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn62r\" (UniqueName: \"kubernetes.io/projected/659b0b50-5f86-4f20-a8b1-88651c23e4a2-kube-api-access-bn62r\") pod \"ovnkube-node-dbgk4\" (UID: \"659b0b50-5f86-4f20-a8b1-88651c23e4a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:39.829967 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.829916 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhlgs\" (UniqueName: \"kubernetes.io/projected/abe4d75b-3d34-4b30-aabe-7952f9debdb4-kube-api-access-jhlgs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:39.830222 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.830145 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8dw2\" (UniqueName: \"kubernetes.io/projected/e5453eb8-3321-4326-9448-debb417eddc8-kube-api-access-k8dw2\") pod \"tuned-hbpv5\" (UID: \"e5453eb8-3321-4326-9448-debb417eddc8\") " pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.830987 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.830934 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrhxw\" (UniqueName: \"kubernetes.io/projected/7e4879e5-9f03-4efa-8875-01f04f0efe05-kube-api-access-jrhxw\") pod \"node-ca-2pvvg\" (UID: \"7e4879e5-9f03-4efa-8875-01f04f0efe05\") " pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.831272 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.831238 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb2rk\" (UniqueName: \"kubernetes.io/projected/9090613d-21a5-4fd0-a284-6503d8fe777e-kube-api-access-kb2rk\") pod \"node-resolver-szwzs\" (UID: \"9090613d-21a5-4fd0-a284-6503d8fe777e\") " pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.832419 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.832397 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snwsw\" (UniqueName: \"kubernetes.io/projected/a8443c5d-e428-4c06-8e61-2e442b129608-kube-api-access-snwsw\") pod \"aws-ebs-csi-driver-node-jdpp4\" (UID: \"a8443c5d-e428-4c06-8e61-2e442b129608\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.905451 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.905416 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-99zkh" Jul 22 16:22:39.913924 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.913892 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" Jul 22 16:22:39.926737 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.926711 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:22:39.934605 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.934582 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" Jul 22 16:22:39.943762 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.943742 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:22:39.944840 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.944823 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" Jul 22 16:22:39.954407 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.954388 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nxhss" Jul 22 16:22:39.964062 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.964040 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-szwzs" Jul 22 16:22:39.972710 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.972693 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pvvg" Jul 22 16:22:39.978545 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:39.978528 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:22:40.221621 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.221544 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:40.221761 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.221698 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:40.221761 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.221753 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret podName:df349bb5-3a16-47f3-80dc-4d13678538c2 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:41.22174068 +0000 UTC m=+3.927183199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret") pod "global-pull-secret-syncer-ckfjr" (UID: "df349bb5-3a16-47f3-80dc-4d13678538c2") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:40.321983 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.321950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:40.322157 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.322115 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:40.322217 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.322185 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs podName:abe4d75b-3d34-4b30-aabe-7952f9debdb4 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:41.322166568 +0000 UTC m=+4.027609091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs") pod "network-metrics-daemon-rw9fk" (UID: "abe4d75b-3d34-4b30-aabe-7952f9debdb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:40.423028 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.423001 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:40.423186 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.423171 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:22:40.423249 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.423196 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:22:40.423249 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.423211 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jjfxx for pod openshift-network-diagnostics/network-check-target-dczqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:40.423351 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:40.423277 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx podName:507c1878-0b1f-4510-acbf-f5ddfae2ca5c nodeName:}" failed. No retries permitted until 2026-07-22 16:22:41.423259058 +0000 UTC m=+4.128701579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jjfxx" (UniqueName: "kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx") pod "network-check-target-dczqj" (UID: "507c1878-0b1f-4510-acbf-f5ddfae2ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:40.605797 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.605762 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b166af_2797_4f86_b009_667c2d19b6b7.slice/crio-da5e582714624aa9aae17a0ea3810f4bebf029b5a3fcf19bde9e6751e4dedfb6 WatchSource:0}: Error finding container da5e582714624aa9aae17a0ea3810f4bebf029b5a3fcf19bde9e6751e4dedfb6: Status 404 returned error can't find the container with id da5e582714624aa9aae17a0ea3810f4bebf029b5a3fcf19bde9e6751e4dedfb6 Jul 22 16:22:40.606343 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.606139 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4879e5_9f03_4efa_8875_01f04f0efe05.slice/crio-482edca2486b9f8677363ef5415e857adb35e475dda6110142c43bedc8a9711e WatchSource:0}: Error finding container 482edca2486b9f8677363ef5415e857adb35e475dda6110142c43bedc8a9711e: Status 404 returned error can't find the container with id 482edca2486b9f8677363ef5415e857adb35e475dda6110142c43bedc8a9711e Jul 22 16:22:40.618725 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.618689 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659b0b50_5f86_4f20_a8b1_88651c23e4a2.slice/crio-36d02656476ece0cb0f64355962b67903d549a48dc6378799c426ecd021ea87e WatchSource:0}: Error finding container 36d02656476ece0cb0f64355962b67903d549a48dc6378799c426ecd021ea87e: Status 404 returned error can't find the container with id 36d02656476ece0cb0f64355962b67903d549a48dc6378799c426ecd021ea87e Jul 22 16:22:40.622807 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.622690 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8443c5d_e428_4c06_8e61_2e442b129608.slice/crio-77b568bfe228acab936818bc7bec7b04ecd12342b8e550da15fe93a7499b1f04 WatchSource:0}: Error finding container 77b568bfe228acab936818bc7bec7b04ecd12342b8e550da15fe93a7499b1f04: Status 404 returned error can't find the container with id 77b568bfe228acab936818bc7bec7b04ecd12342b8e550da15fe93a7499b1f04 Jul 22 16:22:40.623931 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.623573 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9090613d_21a5_4fd0_a284_6503d8fe777e.slice/crio-7813b2cdd3ededd8a5e995c0f14ef94b9f2953c66f84703dedb9174e964300a2 WatchSource:0}: Error finding container 7813b2cdd3ededd8a5e995c0f14ef94b9f2953c66f84703dedb9174e964300a2: Status 404 returned error can't find the container with id 7813b2cdd3ededd8a5e995c0f14ef94b9f2953c66f84703dedb9174e964300a2 Jul 22 16:22:40.624487 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.624466 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5453eb8_3321_4326_9448_debb417eddc8.slice/crio-631c82006e83548a83942514a8df22d44c48efbb9a00d89b1e5c583a7f72ae79 WatchSource:0}: Error finding container 631c82006e83548a83942514a8df22d44c48efbb9a00d89b1e5c583a7f72ae79: Status 404 returned error can't find the container with id 631c82006e83548a83942514a8df22d44c48efbb9a00d89b1e5c583a7f72ae79 Jul 22 16:22:40.625598 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.625581 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c56210_7d70_45ed_bb55_7ca81950c8f4.slice/crio-370deb64d952d265b320f9b54e8bc50746d366f94a46d95acfd1bd40a082a97b WatchSource:0}: Error finding container 370deb64d952d265b320f9b54e8bc50746d366f94a46d95acfd1bd40a082a97b: Status 404 returned error can't find the container with id 370deb64d952d265b320f9b54e8bc50746d366f94a46d95acfd1bd40a082a97b Jul 22 16:22:40.626193 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.626157 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bfdb7e_ccfa_4f28_9958_fd4281974e45.slice/crio-c0f00427497ba57644d5fedea33e92bc0dbd1ce0c74a746e2dca4fdd7fb1e537 WatchSource:0}: Error finding container c0f00427497ba57644d5fedea33e92bc0dbd1ce0c74a746e2dca4fdd7fb1e537: Status 404 returned error can't find the container with id c0f00427497ba57644d5fedea33e92bc0dbd1ce0c74a746e2dca4fdd7fb1e537 Jul 22 16:22:40.628019 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:22:40.627673 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1221eced_0d60_4b12_8249_dbe37c4ba00b.slice/crio-7723d897a378872176d27abf39ad3298e99e9ecd0f07f40db5c45c619118d026 WatchSource:0}: Error finding container 7723d897a378872176d27abf39ad3298e99e9ecd0f07f40db5c45c619118d026: Status 404 returned error can't find the container with id 7723d897a378872176d27abf39ad3298e99e9ecd0f07f40db5c45c619118d026 Jul 22 16:22:40.643919 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.643870 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:17:39 +0000 UTC" deadline="2028-01-08 22:35:22.929312295 +0000 UTC" Jul 22 16:22:40.643919 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.643919 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12846h12m42.28539592s" Jul 22 16:22:40.748363 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.748201 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" event={"ID":"e5453eb8-3321-4326-9448-debb417eddc8","Type":"ContainerStarted","Data":"631c82006e83548a83942514a8df22d44c48efbb9a00d89b1e5c583a7f72ae79"} Jul 22 16:22:40.749233 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.749207 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-szwzs" event={"ID":"9090613d-21a5-4fd0-a284-6503d8fe777e","Type":"ContainerStarted","Data":"7813b2cdd3ededd8a5e995c0f14ef94b9f2953c66f84703dedb9174e964300a2"} Jul 22 16:22:40.750188 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.750162 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pvvg" event={"ID":"7e4879e5-9f03-4efa-8875-01f04f0efe05","Type":"ContainerStarted","Data":"482edca2486b9f8677363ef5415e857adb35e475dda6110142c43bedc8a9711e"} Jul 22 16:22:40.751771 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.751749 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nxhss" event={"ID":"1221eced-0d60-4b12-8249-dbe37c4ba00b","Type":"ContainerStarted","Data":"7723d897a378872176d27abf39ad3298e99e9ecd0f07f40db5c45c619118d026"} Jul 22 16:22:40.753791 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.753766 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-99zkh" event={"ID":"29bfdb7e-ccfa-4f28-9958-fd4281974e45","Type":"ContainerStarted","Data":"c0f00427497ba57644d5fedea33e92bc0dbd1ce0c74a746e2dca4fdd7fb1e537"} Jul 22 16:22:40.756423 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.756397 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49m6s" event={"ID":"12c56210-7d70-45ed-bb55-7ca81950c8f4","Type":"ContainerStarted","Data":"370deb64d952d265b320f9b54e8bc50746d366f94a46d95acfd1bd40a082a97b"} Jul 22 16:22:40.757594 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.757576 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"36d02656476ece0cb0f64355962b67903d549a48dc6378799c426ecd021ea87e"} Jul 22 16:22:40.758651 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.758624 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" event={"ID":"a8443c5d-e428-4c06-8e61-2e442b129608","Type":"ContainerStarted","Data":"77b568bfe228acab936818bc7bec7b04ecd12342b8e550da15fe93a7499b1f04"} Jul 22 16:22:40.759889 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.759866 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerStarted","Data":"da5e582714624aa9aae17a0ea3810f4bebf029b5a3fcf19bde9e6751e4dedfb6"} Jul 22 16:22:40.761771 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.761752 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" event={"ID":"a7300a218e12596f33fabcbb2467c251","Type":"ContainerStarted","Data":"1cdecd9a1bc4458d5e02d43f84179a6661671a7089888ac316492436ee9eb95e"} Jul 22 16:22:40.776461 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:40.776414 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-200.ec2.internal" podStartSLOduration=2.776400097 podStartE2EDuration="2.776400097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:22:40.775524507 +0000 UTC m=+3.480967050" watchObservedRunningTime="2026-07-22 16:22:40.776400097 +0000 UTC m=+3.481842639" Jul 22 16:22:41.230627 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.230533 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:41.230793 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.230729 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:41.230848 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.230795 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret podName:df349bb5-3a16-47f3-80dc-4d13678538c2 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:43.230775435 +0000 UTC m=+5.936217958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret") pod "global-pull-secret-syncer-ckfjr" (UID: "df349bb5-3a16-47f3-80dc-4d13678538c2") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:41.331751 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.331564 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:41.331751 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.331737 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:41.331985 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.331807 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs podName:abe4d75b-3d34-4b30-aabe-7952f9debdb4 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:43.331786478 +0000 UTC m=+6.037229002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs") pod "network-metrics-daemon-rw9fk" (UID: "abe4d75b-3d34-4b30-aabe-7952f9debdb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:41.432274 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.432236 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:41.432462 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.432398 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:22:41.432462 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.432421 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:22:41.432462 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.432433 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jjfxx for pod openshift-network-diagnostics/network-check-target-dczqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:41.432629 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.432493 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx podName:507c1878-0b1f-4510-acbf-f5ddfae2ca5c nodeName:}" failed. No retries permitted until 2026-07-22 16:22:43.432475766 +0000 UTC m=+6.137918289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jjfxx" (UniqueName: "kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx") pod "network-check-target-dczqj" (UID: "507c1878-0b1f-4510-acbf-f5ddfae2ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:41.644140 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.644097 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:17:39 +0000 UTC" deadline="2028-03-18 01:36:25.555921877 +0000 UTC" Jul 22 16:22:41.644140 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.644138 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14505h13m43.911788453s" Jul 22 16:22:41.740679 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.740215 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:41.740679 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.740242 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:41.740679 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.740242 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:41.740679 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.740349 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:41.740679 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.740474 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:41.740679 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:41.740568 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:41.773136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.772152 2555 generic.go:356] "Generic (PLEG): container finished" podID="ec0a202b202e3a58b24c675cdca1845c" containerID="40bef765bd3e70fa27f07103baa186cf99f4750c013c0adf85939102d90e718d" exitCode=0 Jul 22 16:22:41.773136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:41.773044 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" event={"ID":"ec0a202b202e3a58b24c675cdca1845c","Type":"ContainerDied","Data":"40bef765bd3e70fa27f07103baa186cf99f4750c013c0adf85939102d90e718d"} Jul 22 16:22:42.785482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:42.785426 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" event={"ID":"ec0a202b202e3a58b24c675cdca1845c","Type":"ContainerStarted","Data":"f07754008f9fd7be879f5d7000254d2edda0fa89d7868effaa50291d5020c924"} Jul 22 16:22:43.248114 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:43.247999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:43.248283 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.248186 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:43.248283 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.248251 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret podName:df349bb5-3a16-47f3-80dc-4d13678538c2 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:47.248232878 +0000 UTC m=+9.953675401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret") pod "global-pull-secret-syncer-ckfjr" (UID: "df349bb5-3a16-47f3-80dc-4d13678538c2") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:43.348482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:43.348431 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:43.348669 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.348580 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:43.348669 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.348642 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs podName:abe4d75b-3d34-4b30-aabe-7952f9debdb4 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:47.34862448 +0000 UTC m=+10.054067003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs") pod "network-metrics-daemon-rw9fk" (UID: "abe4d75b-3d34-4b30-aabe-7952f9debdb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:43.452434 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:43.452394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:43.452615 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.452562 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:22:43.452615 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.452584 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:22:43.452615 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.452595 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jjfxx for pod openshift-network-diagnostics/network-check-target-dczqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:43.452772 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.452653 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx podName:507c1878-0b1f-4510-acbf-f5ddfae2ca5c nodeName:}" failed. No retries permitted until 2026-07-22 16:22:47.452635746 +0000 UTC m=+10.158078266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jjfxx" (UniqueName: "kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx") pod "network-check-target-dczqj" (UID: "507c1878-0b1f-4510-acbf-f5ddfae2ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:43.741762 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:43.740991 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:43.741762 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.741140 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:43.741762 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:43.741226 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:43.741762 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.741312 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:43.741762 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:43.741361 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:43.741762 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:43.741424 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:45.739587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:45.739185 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:45.739587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:45.739316 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:45.739587 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:45.739416 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:45.739587 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:45.739316 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:45.739587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:45.739188 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:45.739587 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:45.739502 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:47.287015 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:47.286975 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:47.287479 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.287169 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:47.287479 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.287214 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret podName:df349bb5-3a16-47f3-80dc-4d13678538c2 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:55.287201605 +0000 UTC m=+17.992644125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret") pod "global-pull-secret-syncer-ckfjr" (UID: "df349bb5-3a16-47f3-80dc-4d13678538c2") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:47.388159 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:47.388123 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:47.388341 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.388270 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:47.388341 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.388332 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs podName:abe4d75b-3d34-4b30-aabe-7952f9debdb4 nodeName:}" failed. No retries permitted until 2026-07-22 16:22:55.388312752 +0000 UTC m=+18.093755293 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs") pod "network-metrics-daemon-rw9fk" (UID: "abe4d75b-3d34-4b30-aabe-7952f9debdb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:47.489052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:47.489010 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:47.489244 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.489208 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:22:47.489244 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.489226 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:22:47.489244 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.489237 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jjfxx for pod openshift-network-diagnostics/network-check-target-dczqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:47.489410 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.489290 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx podName:507c1878-0b1f-4510-acbf-f5ddfae2ca5c nodeName:}" failed. No retries permitted until 2026-07-22 16:22:55.489272846 +0000 UTC m=+18.194715383 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jjfxx" (UniqueName: "kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx") pod "network-check-target-dczqj" (UID: "507c1878-0b1f-4510-acbf-f5ddfae2ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:47.739950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:47.739868 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:47.740130 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.739967 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:47.740130 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:47.740029 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:47.740130 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.740080 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:47.740356 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:47.740346 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:47.740425 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:47.740397 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:49.739128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:49.738523 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:49.739128 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:49.738648 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:49.739128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:49.738953 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:49.739128 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:49.739024 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:49.739128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:49.739056 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:49.739128 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:49.739116 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:51.739480 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:51.739395 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:51.739851 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:51.739407 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:51.739851 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:51.739542 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:51.739851 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:51.739534 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:51.739851 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:51.739617 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:51.739851 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:51.739683 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:53.738377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:53.738336 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:53.738748 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:53.738472 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:53.738748 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:53.738351 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:53.738748 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:53.738536 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:53.738748 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:53.738336 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:53.738748 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:53.738596 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:55.350134 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:55.350078 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:55.350636 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.350239 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:55.350636 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.350325 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret podName:df349bb5-3a16-47f3-80dc-4d13678538c2 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:11.350304904 +0000 UTC m=+34.055747423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret") pod "global-pull-secret-syncer-ckfjr" (UID: "df349bb5-3a16-47f3-80dc-4d13678538c2") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:22:55.450902 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:55.450857 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:55.451082 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.450985 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:55.451082 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.451055 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs podName:abe4d75b-3d34-4b30-aabe-7952f9debdb4 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:11.451036942 +0000 UTC m=+34.156479480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs") pod "network-metrics-daemon-rw9fk" (UID: "abe4d75b-3d34-4b30-aabe-7952f9debdb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:22:55.551478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:55.551445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:55.551651 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.551582 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:22:55.551651 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.551605 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:22:55.551651 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.551617 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jjfxx for pod openshift-network-diagnostics/network-check-target-dczqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:55.551787 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.551677 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx podName:507c1878-0b1f-4510-acbf-f5ddfae2ca5c nodeName:}" failed. No retries permitted until 2026-07-22 16:23:11.551658415 +0000 UTC m=+34.257100958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jjfxx" (UniqueName: "kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx") pod "network-check-target-dczqj" (UID: "507c1878-0b1f-4510-acbf-f5ddfae2ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:22:55.739316 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:55.739243 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:55.739316 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:55.739243 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:55.739527 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.739383 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:55.739527 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:55.739412 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:55.739639 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.739521 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:55.739639 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:55.739600 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:57.739774 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:57.739316 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:57.739774 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:57.739432 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:57.739774 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:57.739466 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:57.739774 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:57.739518 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:57.739774 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:57.739596 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:57.739774 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:57.739695 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:59.738990 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.738963 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:22:59.742286 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.742263 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:22:59.742389 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:59.742364 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:22:59.742453 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.742439 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:22:59.742528 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:59.742510 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:22:59.742584 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:22:59.742558 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:22:59.823798 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.823557 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pvvg" event={"ID":"7e4879e5-9f03-4efa-8875-01f04f0efe05","Type":"ContainerStarted","Data":"e60259937561a9c320103f8d2165e64f66643f48292323fc34fad63f674d60f4"} Jul 22 16:22:59.829770 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.829732 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-99zkh" event={"ID":"29bfdb7e-ccfa-4f28-9958-fd4281974e45","Type":"ContainerStarted","Data":"9a1ae0ae8e43dee09dce5b8a4c004466be12da2e769d1c20da229e67744c0d87"} Jul 22 16:22:59.836486 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.835753 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" event={"ID":"e5453eb8-3321-4326-9448-debb417eddc8","Type":"ContainerStarted","Data":"728cf832eb2a23368696b2b7fb928b92c4bd060b9c234cc1a259336d12358ec6"} Jul 22 16:22:59.838527 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.837584 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-200.ec2.internal" podStartSLOduration=21.837571703 podStartE2EDuration="21.837571703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:22:42.801933413 +0000 UTC m=+5.507375956" watchObservedRunningTime="2026-07-22 16:22:59.837571703 +0000 UTC m=+22.543014245" Jul 22 16:22:59.853989 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.853897 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2pvvg" podStartSLOduration=12.041164752 podStartE2EDuration="22.853874478s" totalImagesPullingTime="10.812709726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:22:59.837626924 +0000 UTC m=+22.543069470" watchObservedRunningTime="2026-07-22 16:22:59.853874478 +0000 UTC m=+22.559317023" Jul 22 16:22:59.854928 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.854530 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hbpv5" podStartSLOduration=3.8241767639999997 podStartE2EDuration="22.854518119s" totalImagesPullingTime="19.030341355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:22:59.853562055 +0000 UTC m=+22.559004598" watchObservedRunningTime="2026-07-22 16:22:59.854518119 +0000 UTC m=+22.559960662" Jul 22 16:22:59.870990 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:22:59.870758 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-99zkh" podStartSLOduration=3.7894627119999997 podStartE2EDuration="22.870738221s" totalImagesPullingTime="19.081275509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:22:59.870361406 +0000 UTC m=+22.575803948" watchObservedRunningTime="2026-07-22 16:22:59.870738221 +0000 UTC m=+22.576180765" Jul 22 16:23:00.839961 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.839735 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nxhss" event={"ID":"1221eced-0d60-4b12-8249-dbe37c4ba00b","Type":"ContainerStarted","Data":"2bae76beb7185b8ab4dce3da38dcfa7cc760f53bd45958ed628f2128f9ddba0a"} Jul 22 16:23:00.841067 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.841047 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-49m6s" event={"ID":"12c56210-7d70-45ed-bb55-7ca81950c8f4","Type":"ContainerStarted","Data":"b490023df7d42ec4f23a8a67ff2aa2f731c7b3b1fc936768632788b83203ff5b"} Jul 22 16:23:00.843816 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.843796 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"ea3910b741adab7fc147d9fa734d1abf55e8dff48545b864923004042cbc38eb"} Jul 22 16:23:00.843950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.843850 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"10c47de39fc035cc7a56aeeed8577f400130d231fc36bde45ef5521ac482af1e"} Jul 22 16:23:00.843950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.843881 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"44377122135d82cfdc98f3ea67d7bd5a3e2282169cd44a3aa218eb45210b89f6"} Jul 22 16:23:00.843950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.843919 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"19a39fb705cb4e03a041df19901fdbeba1021f7c6b478c5af85c43bde6680744"} Jul 22 16:23:00.843950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.843948 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"4db923a32fd527da6905521fed66704ad00626a786230479d5958d1c250a7733"} Jul 22 16:23:00.844117 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.843974 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"96bd7e71041934c1d8c43a4b643ce327dacc794021f94536492777b1133ccd44"} Jul 22 16:23:00.845205 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.845180 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" event={"ID":"a8443c5d-e428-4c06-8e61-2e442b129608","Type":"ContainerStarted","Data":"9ca40982152828d098b414ea7912260a325a3b2fcdacc421e08515bd57852fd6"} Jul 22 16:23:00.846611 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.846591 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b166af-2797-4f86-b009-667c2d19b6b7" containerID="b9e73ee84452d49a1d4af47b013a56ece66793251b56c2219863d2fbd6b4dbcf" exitCode=0 Jul 22 16:23:00.846677 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.846664 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerDied","Data":"b9e73ee84452d49a1d4af47b013a56ece66793251b56c2219863d2fbd6b4dbcf"} Jul 22 16:23:00.848051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.847927 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-szwzs" event={"ID":"9090613d-21a5-4fd0-a284-6503d8fe777e","Type":"ContainerStarted","Data":"c8a13d8c601c5b702fa8efd66586cf58f5ac49e43c5d03d451ca5fa478f1276f"} Jul 22 16:23:00.856010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.855970 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-49m6s" podStartSLOduration=4.8306297780000005 podStartE2EDuration="23.855959341s" totalImagesPullingTime="19.025329563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:00.855476109 +0000 UTC m=+23.560918651" watchObservedRunningTime="2026-07-22 16:23:00.855959341 +0000 UTC m=+23.561401882" Jul 22 16:23:00.889972 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:00.889930 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-szwzs" podStartSLOduration=4.860176634 podStartE2EDuration="23.889915865s" totalImagesPullingTime="19.029739231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:00.889350201 +0000 UTC m=+23.594792743" watchObservedRunningTime="2026-07-22 16:23:00.889915865 +0000 UTC m=+23.595358406" Jul 22 16:23:01.122797 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.122773 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 16:23:01.680755 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.680644 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T16:23:01.122793946Z","UUID":"23828a2c-25f8-4e2a-9694-8be87848fa56","Handler":null,"Name":"","Endpoint":""} Jul 22 16:23:01.682480 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.682457 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 16:23:01.682613 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.682491 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 16:23:01.738710 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.738675 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:01.738869 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:01.738796 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:23:01.739226 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.739200 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:01.739348 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:01.739309 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:23:01.739407 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.739365 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:01.739464 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:01.739426 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:23:01.854080 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.854040 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" event={"ID":"a8443c5d-e428-4c06-8e61-2e442b129608","Type":"ContainerStarted","Data":"9e5df5ef65f141a52897eb9ee5bc0c07b2d44b3c76d68b55acf6c173a2bfd7f8"} Jul 22 16:23:01.869575 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:01.869529 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nxhss" podStartSLOduration=5.847001164 podStartE2EDuration="24.869512574s" totalImagesPullingTime="19.02251141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:01.869134435 +0000 UTC m=+24.574576980" watchObservedRunningTime="2026-07-22 16:23:01.869512574 +0000 UTC m=+24.574955116" Jul 22 16:23:02.859375 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:02.859336 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"75cba539d701908181525f123db36ac52909628c3f7b86ac096fd81124af8751"} Jul 22 16:23:02.861954 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:02.861925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" event={"ID":"a8443c5d-e428-4c06-8e61-2e442b129608","Type":"ContainerStarted","Data":"c16489a41bdec72738f7665400999ab463a4b82f0e19de30d274dd23ef4157c8"} Jul 22 16:23:02.881137 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:02.881077 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdpp4" podStartSLOduration=4.795520282 podStartE2EDuration="25.881063446s" totalImagesPullingTime="21.085543164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:02.880633047 +0000 UTC m=+25.586075591" watchObservedRunningTime="2026-07-22 16:23:02.881063446 +0000 UTC m=+25.586506011" Jul 22 16:23:03.738834 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:03.738791 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:03.739030 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:03.738843 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:03.739030 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:03.738791 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:03.739030 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:03.738939 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:23:03.739224 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:03.739023 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:23:03.739224 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:03.739110 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:23:04.401452 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:04.401413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:04.487787 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:04.487737 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:05.739330 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.739164 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:05.740066 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.739164 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:05.740066 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:05.739411 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:23:05.740066 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.739167 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:05.740066 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:05.739470 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:23:05.740066 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:05.739567 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:23:05.818772 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.818726 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:23:05.819432 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.819414 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:23:05.869873 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.869832 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b166af-2797-4f86-b009-667c2d19b6b7" containerID="c01a25420110ac7bd9a01d69c00c31b74f769833d44b80deaa07feb0efb03a1a" exitCode=0 Jul 22 16:23:05.870041 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.869891 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerDied","Data":"c01a25420110ac7bd9a01d69c00c31b74f769833d44b80deaa07feb0efb03a1a"} Jul 22 16:23:05.873684 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.873652 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" event={"ID":"659b0b50-5f86-4f20-a8b1-88651c23e4a2","Type":"ContainerStarted","Data":"755009b87998f35f9a899c95c0f6bc5910373d42c685571b496792247fd1e5fe"} Jul 22 16:23:05.873905 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.873859 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:23:05.874383 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.874367 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-49m6s" Jul 22 16:23:05.930855 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.930763 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" podStartSLOduration=9.826208339 podStartE2EDuration="28.930747276s" totalImagesPullingTime="19.104538937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:05.929001794 +0000 UTC m=+28.634444337" watchObservedRunningTime="2026-07-22 16:23:05.930747276 +0000 UTC m=+28.636189817" Jul 22 16:23:05.951320 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:05.951290 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:06.877680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.877456 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerStarted","Data":"deeace372d463217d084c7dae4260fbb13c816e6582ab1ba3f6d5c1381bf52c3"} Jul 22 16:23:06.878148 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.878126 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:23:06.878365 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.878205 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:23:06.878365 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.878244 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:23:06.899315 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.899285 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:23:06.908513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.908482 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:06.910516 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.910498 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:23:06.972189 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:06.972139 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:07.199426 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.199392 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rw9fk"] Jul 22 16:23:07.199636 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.199575 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:07.199723 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:07.199700 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:23:07.200186 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.200155 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ckfjr"] Jul 22 16:23:07.200286 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.200272 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:07.200618 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:07.200370 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:23:07.200756 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.200739 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dczqj"] Jul 22 16:23:07.200825 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.200815 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:07.200905 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:07.200886 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:23:07.881449 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.881412 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b166af-2797-4f86-b009-667c2d19b6b7" containerID="deeace372d463217d084c7dae4260fbb13c816e6582ab1ba3f6d5c1381bf52c3" exitCode=0 Jul 22 16:23:07.881914 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:07.881496 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerDied","Data":"deeace372d463217d084c7dae4260fbb13c816e6582ab1ba3f6d5c1381bf52c3"} Jul 22 16:23:08.001472 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:08.001443 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:08.102283 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:08.102251 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:08.703360 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:08.703334 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:08.738789 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:08.738762 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:08.738967 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:08.738762 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:08.738967 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:08.738901 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:08.738967 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:08.738898 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:23:08.739142 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:08.738998 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:23:08.739142 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:08.739063 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:23:08.804595 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:08.804558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:09.438568 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:09.438399 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:09.538441 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:09.538397 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:09.620715 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:09.620690 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:09.679920 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:09.679885 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:09.888314 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:09.888275 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b166af-2797-4f86-b009-667c2d19b6b7" containerID="24633cd6f335029b810ffb26d07ef66150783af36e2b80cb517d2dd845573cb4" exitCode=0 Jul 22 16:23:09.888474 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:09.888317 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerDied","Data":"24633cd6f335029b810ffb26d07ef66150783af36e2b80cb517d2dd845573cb4"} Jul 22 16:23:10.739236 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:10.738720 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:10.739236 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:10.738753 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:10.739236 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:10.738840 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dczqj" podUID="507c1878-0b1f-4510-acbf-f5ddfae2ca5c" Jul 22 16:23:10.739236 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:10.738878 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:10.739236 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:10.739002 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ckfjr" podUID="df349bb5-3a16-47f3-80dc-4d13678538c2" Jul 22 16:23:10.739236 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:10.739070 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rw9fk" podUID="abe4d75b-3d34-4b30-aabe-7952f9debdb4" Jul 22 16:23:11.377238 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:11.377213 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:11.377419 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.377393 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:23:11.377479 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.377471 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret podName:df349bb5-3a16-47f3-80dc-4d13678538c2 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:43.377449432 +0000 UTC m=+66.082891981 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret") pod "global-pull-secret-syncer-ckfjr" (UID: "df349bb5-3a16-47f3-80dc-4d13678538c2") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:23:11.478433 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:11.478393 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:11.478585 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.478567 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:23:11.478663 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.478640 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs podName:abe4d75b-3d34-4b30-aabe-7952f9debdb4 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:43.478621903 +0000 UTC m=+66.184064438 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs") pod "network-metrics-daemon-rw9fk" (UID: "abe4d75b-3d34-4b30-aabe-7952f9debdb4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:23:11.579749 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:11.579709 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:11.579906 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.579841 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:23:11.579906 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.579858 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:23:11.579906 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.579870 2555 projected.go:196] Error preparing data for projected volume kube-api-access-jjfxx for pod openshift-network-diagnostics/network-check-target-dczqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:23:11.580007 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:11.579918 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx podName:507c1878-0b1f-4510-acbf-f5ddfae2ca5c nodeName:}" failed. No retries permitted until 2026-07-22 16:23:43.579904882 +0000 UTC m=+66.285347401 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jjfxx" (UniqueName: "kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx") pod "network-check-target-dczqj" (UID: "507c1878-0b1f-4510-acbf-f5ddfae2ca5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:23:12.586533 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.586498 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-200.ec2.internal" event="NodeReady" Jul 22 16:23:12.587048 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.586663 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 16:23:12.622426 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.622385 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-694d669fcc-wv78c"] Jul 22 16:23:12.627211 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.627177 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.630740 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.630718 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 16:23:12.630877 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.630778 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 16:23:12.630877 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.630832 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kgbkt\"" Jul 22 16:23:12.630877 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.630781 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 16:23:12.635237 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.635210 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g6gmg"] Jul 22 16:23:12.638592 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.638572 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7mrnv"] Jul 22 16:23:12.638744 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.638724 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.639108 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.639066 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 16:23:12.641807 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.641789 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cfcr6\"" Jul 22 16:23:12.641923 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.641793 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 16:23:12.641923 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.641793 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 16:23:12.642278 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.642261 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-694d669fcc-wv78c"] Jul 22 16:23:12.642371 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.642358 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:12.646610 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.645535 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 16:23:12.646610 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.645535 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8zl58\"" Jul 22 16:23:12.646610 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.645559 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 16:23:12.646610 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.645613 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 16:23:12.646610 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.646576 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g6gmg"] Jul 22 16:23:12.650064 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.650042 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7mrnv"] Jul 22 16:23:12.738487 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.738451 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:12.738685 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.738455 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:12.738685 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.738470 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:12.741552 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.741529 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:23:12.741675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.741553 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 16:23:12.741675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.741563 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 16:23:12.741675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.741570 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:23:12.741675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.741571 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h7v2l\"" Jul 22 16:23:12.741675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.741540 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vhlm5\"" Jul 22 16:23:12.753166 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.753142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:12.789033 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.788995 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.789207 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789055 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-image-registry-private-configuration\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.789207 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789100 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-installation-pull-secrets\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.789207 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxkhj\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-kube-api-access-kxkhj\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.789355 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789231 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0618f128-da4f-48f9-80f8-2556651f28e4-cert\") pod \"ingress-canary-7mrnv\" (UID: \"0618f128-da4f-48f9-80f8-2556651f28e4\") " pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:12.789355 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789277 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46d4b72d-b63d-4e54-a782-abfdd53f3c48-config-volume\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.789355 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789304 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-registry-certificates\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.789482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789361 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4b72d-b63d-4e54-a782-abfdd53f3c48-tmp-dir\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.789482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789422 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsspk\" (UniqueName: \"kubernetes.io/projected/46d4b72d-b63d-4e54-a782-abfdd53f3c48-kube-api-access-xsspk\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.789482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789462 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.789619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg4x8\" (UniqueName: \"kubernetes.io/projected/0618f128-da4f-48f9-80f8-2556651f28e4-kube-api-access-jg4x8\") pod \"ingress-canary-7mrnv\" (UID: \"0618f128-da4f-48f9-80f8-2556651f28e4\") " pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:12.789619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789549 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbc42118-83c7-492f-974d-876c8954257e-ca-trust-extracted\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.790009 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.789600 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-trusted-ca\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.790167 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.790036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-bound-sa-token\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.818434 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.818393 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:12.891324 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891245 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4b72d-b63d-4e54-a782-abfdd53f3c48-tmp-dir\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.891324 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891289 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsspk\" (UniqueName: \"kubernetes.io/projected/46d4b72d-b63d-4e54-a782-abfdd53f3c48-kube-api-access-xsspk\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.891324 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.891593 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891359 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg4x8\" (UniqueName: \"kubernetes.io/projected/0618f128-da4f-48f9-80f8-2556651f28e4-kube-api-access-jg4x8\") pod \"ingress-canary-7mrnv\" (UID: \"0618f128-da4f-48f9-80f8-2556651f28e4\") " pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:12.891593 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891382 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbc42118-83c7-492f-974d-876c8954257e-ca-trust-extracted\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.891593 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891404 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-trusted-ca\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.891593 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:12.891533 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:23:12.891593 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:12.891557 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-694d669fcc-wv78c: secret "image-registry-tls" not found Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:12.891622 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls podName:bbc42118-83c7-492f-974d-876c8954257e nodeName:}" failed. No retries permitted until 2026-07-22 16:23:13.391600594 +0000 UTC m=+36.097043127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls") pod "image-registry-694d669fcc-wv78c" (UID: "bbc42118-83c7-492f-974d-876c8954257e") : secret "image-registry-tls" not found Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891647 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-bound-sa-token\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891656 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4b72d-b63d-4e54-a782-abfdd53f3c48-tmp-dir\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891695 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-image-registry-private-configuration\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891756 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-installation-pull-secrets\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891759 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbc42118-83c7-492f-974d-876c8954257e-ca-trust-extracted\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.891828 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:12.891788 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:23:12.892228 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:12.891842 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls podName:46d4b72d-b63d-4e54-a782-abfdd53f3c48 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:13.391824477 +0000 UTC m=+36.097267000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls") pod "dns-default-g6gmg" (UID: "46d4b72d-b63d-4e54-a782-abfdd53f3c48") : secret "dns-default-metrics-tls" not found Jul 22 16:23:12.892228 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891793 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxkhj\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-kube-api-access-kxkhj\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.892228 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891897 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0618f128-da4f-48f9-80f8-2556651f28e4-cert\") pod \"ingress-canary-7mrnv\" (UID: \"0618f128-da4f-48f9-80f8-2556651f28e4\") " pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:12.892228 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891931 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46d4b72d-b63d-4e54-a782-abfdd53f3c48-config-volume\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.892228 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.891956 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-registry-certificates\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.892470 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.892389 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-trusted-ca\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.892597 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.892571 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46d4b72d-b63d-4e54-a782-abfdd53f3c48-config-volume\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.892865 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.892841 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-registry-certificates\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.896281 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.896253 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-installation-pull-secrets\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.896388 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.896371 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0618f128-da4f-48f9-80f8-2556651f28e4-cert\") pod \"ingress-canary-7mrnv\" (UID: \"0618f128-da4f-48f9-80f8-2556651f28e4\") " pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:12.896429 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.896408 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-image-registry-private-configuration\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.900944 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.900923 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsspk\" (UniqueName: \"kubernetes.io/projected/46d4b72d-b63d-4e54-a782-abfdd53f3c48-kube-api-access-xsspk\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:12.901035 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.900980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxkhj\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-kube-api-access-kxkhj\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.901074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.901055 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-bound-sa-token\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:12.901440 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.901416 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg4x8\" (UniqueName: \"kubernetes.io/projected/0618f128-da4f-48f9-80f8-2556651f28e4-kube-api-access-jg4x8\") pod \"ingress-canary-7mrnv\" (UID: \"0618f128-da4f-48f9-80f8-2556651f28e4\") " pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:12.956572 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:12.956535 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7mrnv" Jul 22 16:23:13.396695 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:13.396659 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:13.396875 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:13.396717 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:13.396875 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:13.396831 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:23:13.396875 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:13.396860 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-694d669fcc-wv78c: secret "image-registry-tls" not found Jul 22 16:23:13.396875 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:13.396835 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:23:13.397059 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:13.396922 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls podName:46d4b72d-b63d-4e54-a782-abfdd53f3c48 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:14.396909442 +0000 UTC m=+37.102351961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls") pod "dns-default-g6gmg" (UID: "46d4b72d-b63d-4e54-a782-abfdd53f3c48") : secret "dns-default-metrics-tls" not found Jul 22 16:23:13.397059 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:13.396936 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls podName:bbc42118-83c7-492f-974d-876c8954257e nodeName:}" failed. No retries permitted until 2026-07-22 16:23:14.396930323 +0000 UTC m=+37.102372843 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls") pod "image-registry-694d669fcc-wv78c" (UID: "bbc42118-83c7-492f-974d-876c8954257e") : secret "image-registry-tls" not found Jul 22 16:23:13.946358 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:13.946221 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7mrnv"] Jul 22 16:23:13.950303 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:13.950278 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0618f128_da4f_48f9_80f8_2556651f28e4.slice/crio-c0d75af1dbdcaedbd3d9b83e7c2acefa5449ff3dcb45c59237825a3267957c7d WatchSource:0}: Error finding container c0d75af1dbdcaedbd3d9b83e7c2acefa5449ff3dcb45c59237825a3267957c7d: Status 404 returned error can't find the container with id c0d75af1dbdcaedbd3d9b83e7c2acefa5449ff3dcb45c59237825a3267957c7d Jul 22 16:23:14.405078 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:14.404989 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:14.405078 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:14.405054 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:14.405282 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:14.405176 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:23:14.405282 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:14.405194 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-694d669fcc-wv78c: secret "image-registry-tls" not found Jul 22 16:23:14.405282 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:14.405200 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:23:14.405282 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:14.405248 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls podName:46d4b72d-b63d-4e54-a782-abfdd53f3c48 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:16.405233409 +0000 UTC m=+39.110675932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls") pod "dns-default-g6gmg" (UID: "46d4b72d-b63d-4e54-a782-abfdd53f3c48") : secret "dns-default-metrics-tls" not found Jul 22 16:23:14.405282 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:14.405264 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls podName:bbc42118-83c7-492f-974d-876c8954257e nodeName:}" failed. No retries permitted until 2026-07-22 16:23:16.405255729 +0000 UTC m=+39.110698249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls") pod "image-registry-694d669fcc-wv78c" (UID: "bbc42118-83c7-492f-974d-876c8954257e") : secret "image-registry-tls" not found Jul 22 16:23:14.748596 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:14.748516 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:14.808994 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:14.808947 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:14.904498 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:14.904464 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b166af-2797-4f86-b009-667c2d19b6b7" containerID="22dd2c1535e54c5ddefa7179923905077f3e7a063840ae25191c24532bedeeaf" exitCode=0 Jul 22 16:23:14.904665 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:14.904562 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerDied","Data":"22dd2c1535e54c5ddefa7179923905077f3e7a063840ae25191c24532bedeeaf"} Jul 22 16:23:14.905953 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:14.905926 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7mrnv" event={"ID":"0618f128-da4f-48f9-80f8-2556651f28e4","Type":"ContainerStarted","Data":"c0d75af1dbdcaedbd3d9b83e7c2acefa5449ff3dcb45c59237825a3267957c7d"} Jul 22 16:23:15.727024 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:15.726992 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:15.785329 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:15.785297 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:15.912036 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:15.911994 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b166af-2797-4f86-b009-667c2d19b6b7" containerID="c17d8a31181e2b0ad3726f49c76c8f12f8ea9c0fb495e3bde66b66442f5d370e" exitCode=0 Jul 22 16:23:15.912229 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:15.912057 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerDied","Data":"c17d8a31181e2b0ad3726f49c76c8f12f8ea9c0fb495e3bde66b66442f5d370e"} Jul 22 16:23:16.421587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.421548 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:16.421778 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.421645 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:16.421778 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:16.421725 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:23:16.421778 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:16.421748 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-694d669fcc-wv78c: secret "image-registry-tls" not found Jul 22 16:23:16.421778 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:16.421751 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:23:16.421969 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:16.421839 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls podName:46d4b72d-b63d-4e54-a782-abfdd53f3c48 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:20.421817028 +0000 UTC m=+43.127259566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls") pod "dns-default-g6gmg" (UID: "46d4b72d-b63d-4e54-a782-abfdd53f3c48") : secret "dns-default-metrics-tls" not found Jul 22 16:23:16.421969 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:16.421858 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls podName:bbc42118-83c7-492f-974d-876c8954257e nodeName:}" failed. No retries permitted until 2026-07-22 16:23:20.4218516 +0000 UTC m=+43.127294119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls") pod "image-registry-694d669fcc-wv78c" (UID: "bbc42118-83c7-492f-974d-876c8954257e") : secret "image-registry-tls" not found Jul 22 16:23:16.877482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.877456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:16.916056 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.916028 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7mrnv" event={"ID":"0618f128-da4f-48f9-80f8-2556651f28e4","Type":"ContainerStarted","Data":"245e0422195b74bb8c88ac830147ec9bd09fac48975015b8673961408b6cc766"} Jul 22 16:23:16.919340 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.919309 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" event={"ID":"f7b166af-2797-4f86-b009-667c2d19b6b7","Type":"ContainerStarted","Data":"52df446026bc060a49a619fd405f27f6f5616cdf7691eea9ac48efb84712964d"} Jul 22 16:23:16.934387 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.934331 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7mrnv" podStartSLOduration=2.078047831 podStartE2EDuration="4.934310141s" totalImagesPullingTime="2.85626231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:16.933073907 +0000 UTC m=+39.638516448" watchObservedRunningTime="2026-07-22 16:23:16.934310141 +0000 UTC m=+39.639752684" Jul 22 16:23:16.947005 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.946928 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:16.958571 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:16.958520 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n2wx6" podStartSLOduration=10.870046024 podStartE2EDuration="39.958505051s" totalImagesPullingTime="29.088459027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:16.95652322 +0000 UTC m=+39.661965763" watchObservedRunningTime="2026-07-22 16:23:16.958505051 +0000 UTC m=+39.663947592" Jul 22 16:23:17.095509 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:17.095473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:17.163792 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:17.163760 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:17.377476 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:17.377450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:17.476962 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:17.476929 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:18.080959 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.080925 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:18.114949 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.114918 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8"] Jul 22 16:23:18.135609 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.135585 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8"] Jul 22 16:23:18.135724 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.135673 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" Jul 22 16:23:18.138669 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.138650 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 22 16:23:18.138669 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.138654 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 22 16:23:18.139691 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.139671 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-cdvjv\"" Jul 22 16:23:18.156939 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.156906 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:18.208920 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.208887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:18.237029 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.237007 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jz58\" (UniqueName: \"kubernetes.io/projected/24e58fae-3f60-4a62-ae19-2d3a7396628c-kube-api-access-4jz58\") pod \"migrator-7cc9c7ffd7-ndvc8\" (UID: \"24e58fae-3f60-4a62-ae19-2d3a7396628c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" Jul 22 16:23:18.338428 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.338346 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz58\" (UniqueName: \"kubernetes.io/projected/24e58fae-3f60-4a62-ae19-2d3a7396628c-kube-api-access-4jz58\") pod \"migrator-7cc9c7ffd7-ndvc8\" (UID: \"24e58fae-3f60-4a62-ae19-2d3a7396628c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" Jul 22 16:23:18.349912 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.349887 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jz58\" (UniqueName: \"kubernetes.io/projected/24e58fae-3f60-4a62-ae19-2d3a7396628c-kube-api-access-4jz58\") pod \"migrator-7cc9c7ffd7-ndvc8\" (UID: \"24e58fae-3f60-4a62-ae19-2d3a7396628c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" Jul 22 16:23:18.444899 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.444869 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" Jul 22 16:23:18.590905 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.590835 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qvpmq"] Jul 22 16:23:18.597298 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:18.597276 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e58fae_3f60_4a62_ae19_2d3a7396628c.slice/crio-253ee23e7cdb6940d34f5a6a4bc7e0301085a230a1c57c0949653b8c0cb28819 WatchSource:0}: Error finding container 253ee23e7cdb6940d34f5a6a4bc7e0301085a230a1c57c0949653b8c0cb28819: Status 404 returned error can't find the container with id 253ee23e7cdb6940d34f5a6a4bc7e0301085a230a1c57c0949653b8c0cb28819 Jul 22 16:23:18.614489 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.614465 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:18.626365 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.626338 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8"] Jul 22 16:23:18.626471 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.626377 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qvpmq"] Jul 22 16:23:18.626471 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.626458 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.629250 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.629230 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 16:23:18.629338 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.629292 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 16:23:18.629477 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.629462 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 16:23:18.629515 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.629499 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 16:23:18.629839 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.629818 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qp2c5\"" Jul 22 16:23:18.641134 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.641113 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abbf1ead-2639-429c-8a3b-eb8664f3996d-crio-socket\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.641227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.641141 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.641227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.641180 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkclz\" (UniqueName: \"kubernetes.io/projected/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-api-access-rkclz\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.641227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.641207 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abbf1ead-2639-429c-8a3b-eb8664f3996d-data-volume\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.641227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.641224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.680478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.680445 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:18.741627 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.741590 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abbf1ead-2639-429c-8a3b-eb8664f3996d-crio-socket\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.741627 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.741626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.741823 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.741652 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkclz\" (UniqueName: \"kubernetes.io/projected/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-api-access-rkclz\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.741823 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.741709 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abbf1ead-2639-429c-8a3b-eb8664f3996d-data-volume\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.741823 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:18.741731 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:23:18.741823 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.741751 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.741823 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:18.741797 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-rbac-proxy-cm podName:abbf1ead-2639-429c-8a3b-eb8664f3996d nodeName:}" failed. No retries permitted until 2026-07-22 16:23:19.241776618 +0000 UTC m=+41.947219139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qvpmq" (UID: "abbf1ead-2639-429c-8a3b-eb8664f3996d") : configmap "kube-rbac-proxy" not found Jul 22 16:23:18.741823 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.741816 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abbf1ead-2639-429c-8a3b-eb8664f3996d-crio-socket\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.742078 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:18.741854 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:23:18.742078 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:18.741922 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls podName:abbf1ead-2639-429c-8a3b-eb8664f3996d nodeName:}" failed. No retries permitted until 2026-07-22 16:23:19.241905673 +0000 UTC m=+41.947348194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qvpmq" (UID: "abbf1ead-2639-429c-8a3b-eb8664f3996d") : secret "insights-runtime-extractor-tls" not found Jul 22 16:23:18.742078 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.742063 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abbf1ead-2639-429c-8a3b-eb8664f3996d-data-volume\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.749816 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.749797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkclz\" (UniqueName: \"kubernetes.io/projected/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-api-access-rkclz\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:18.924829 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:18.924746 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" event={"ID":"24e58fae-3f60-4a62-ae19-2d3a7396628c","Type":"ContainerStarted","Data":"253ee23e7cdb6940d34f5a6a4bc7e0301085a230a1c57c0949653b8c0cb28819"} Jul 22 16:23:19.245334 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.245249 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:19.245742 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.245377 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:19.245742 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:19.245422 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:23:19.245742 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:19.245501 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls podName:abbf1ead-2639-429c-8a3b-eb8664f3996d nodeName:}" failed. No retries permitted until 2026-07-22 16:23:20.245481432 +0000 UTC m=+42.950923966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qvpmq" (UID: "abbf1ead-2639-429c-8a3b-eb8664f3996d") : secret "insights-runtime-extractor-tls" not found Jul 22 16:23:19.245917 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.245896 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abbf1ead-2639-429c-8a3b-eb8664f3996d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:19.351844 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.351808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:19.682833 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.682796 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-6fd7n"] Jul 22 16:23:19.701355 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.701321 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-6fd7n"] Jul 22 16:23:19.701493 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.701361 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.704213 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.704192 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 16:23:19.704326 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.704193 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 16:23:19.704326 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.704201 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 16:23:19.705368 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.705352 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 16:23:19.705467 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.705449 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bc92d\"" Jul 22 16:23:19.721594 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.721567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:19.748822 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.748795 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-signing-key\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.748987 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.748845 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-signing-cabundle\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.748987 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.748975 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrqtb\" (UniqueName: \"kubernetes.io/projected/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-kube-api-access-hrqtb\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.783438 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.783402 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:19.849368 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.849330 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-signing-cabundle\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.849535 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.849454 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrqtb\" (UniqueName: \"kubernetes.io/projected/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-kube-api-access-hrqtb\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.849535 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.849530 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-signing-key\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.850062 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.850036 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-signing-cabundle\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.852263 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.852244 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-signing-key\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:19.857798 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:19.857773 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrqtb\" (UniqueName: \"kubernetes.io/projected/a4bbf97d-b94c-4d94-924c-66d7bf5b40fe-kube-api-access-hrqtb\") pod \"service-ca-58598695f7-6fd7n\" (UID: \"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe\") " pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:20.009814 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.009747 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-6fd7n" Jul 22 16:23:20.175033 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.175004 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-6fd7n"] Jul 22 16:23:20.178148 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:20.178122 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4bbf97d_b94c_4d94_924c_66d7bf5b40fe.slice/crio-81663734adedeb67e460fbe589e99c38278fcd412dda3d05786d245673f5f000 WatchSource:0}: Error finding container 81663734adedeb67e460fbe589e99c38278fcd412dda3d05786d245673f5f000: Status 404 returned error can't find the container with id 81663734adedeb67e460fbe589e99c38278fcd412dda3d05786d245673f5f000 Jul 22 16:23:20.252179 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.252142 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:20.252552 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:20.252302 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:23:20.252552 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:20.252362 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls podName:abbf1ead-2639-429c-8a3b-eb8664f3996d nodeName:}" failed. No retries permitted until 2026-07-22 16:23:22.252346284 +0000 UTC m=+44.957788825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qvpmq" (UID: "abbf1ead-2639-429c-8a3b-eb8664f3996d") : secret "insights-runtime-extractor-tls" not found Jul 22 16:23:20.454098 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.454061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:20.454218 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.454136 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:20.454218 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:20.454201 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:23:20.454218 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:20.454218 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-694d669fcc-wv78c: secret "image-registry-tls" not found Jul 22 16:23:20.454373 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:20.454273 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls podName:bbc42118-83c7-492f-974d-876c8954257e nodeName:}" failed. No retries permitted until 2026-07-22 16:23:28.454259353 +0000 UTC m=+51.159701873 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls") pod "image-registry-694d669fcc-wv78c" (UID: "bbc42118-83c7-492f-974d-876c8954257e") : secret "image-registry-tls" not found Jul 22 16:23:20.454373 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:20.454288 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:23:20.454373 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:20.454344 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls podName:46d4b72d-b63d-4e54-a782-abfdd53f3c48 nodeName:}" failed. No retries permitted until 2026-07-22 16:23:28.45432808 +0000 UTC m=+51.159770613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls") pod "dns-default-g6gmg" (UID: "46d4b72d-b63d-4e54-a782-abfdd53f3c48") : secret "dns-default-metrics-tls" not found Jul 22 16:23:20.588276 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.588244 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:20.934120 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.933138 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" event={"ID":"24e58fae-3f60-4a62-ae19-2d3a7396628c","Type":"ContainerStarted","Data":"257669623c880c9add040ca8757576c6950e97e3780a02de06d216a3ddcc4985"} Jul 22 16:23:20.934120 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.933251 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" event={"ID":"24e58fae-3f60-4a62-ae19-2d3a7396628c","Type":"ContainerStarted","Data":"b81ef84f73a2e2cd4f629ef245983a7c89f1dc54d4f15e54cafdd530527f4a85"} Jul 22 16:23:20.934686 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.934649 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-6fd7n" event={"ID":"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe","Type":"ContainerStarted","Data":"81663734adedeb67e460fbe589e99c38278fcd412dda3d05786d245673f5f000"} Jul 22 16:23:20.950710 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:20.950648 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ndvc8" podStartSLOduration=1.16273068 podStartE2EDuration="2.95063158s" totalImagesPullingTime="1.7879009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:20.950209548 +0000 UTC m=+43.655652091" watchObservedRunningTime="2026-07-22 16:23:20.95063158 +0000 UTC m=+43.656074121" Jul 22 16:23:21.186001 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:21.185928 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-szwzs_9090613d-21a5-4fd0-a284-6503d8fe777e/dns-node-resolver/0.log" Jul 22 16:23:21.783969 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:21.783940 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pvvg_7e4879e5-9f03-4efa-8875-01f04f0efe05/node-ca/0.log" Jul 22 16:23:22.268559 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:22.268485 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:22.268696 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:22.268636 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:23:22.268730 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:22.268704 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls podName:abbf1ead-2639-429c-8a3b-eb8664f3996d nodeName:}" failed. No retries permitted until 2026-07-22 16:23:26.268688332 +0000 UTC m=+48.974130852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qvpmq" (UID: "abbf1ead-2639-429c-8a3b-eb8664f3996d") : secret "insights-runtime-extractor-tls" not found Jul 22 16:23:22.784697 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:22.784667 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7mrnv_0618f128-da4f-48f9-80f8-2556651f28e4/serve-healthcheck-canary/0.log" Jul 22 16:23:22.941058 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:22.941028 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-6fd7n" event={"ID":"a4bbf97d-b94c-4d94-924c-66d7bf5b40fe","Type":"ContainerStarted","Data":"e52c2d7b8b9809c2ce165a14732f396a4ee7c92a9bb2955b1e583b60454ce097"} Jul 22 16:23:22.984710 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:22.984681 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ndvc8_24e58fae-3f60-4a62-ae19-2d3a7396628c/migrator/0.log" Jul 22 16:23:23.184202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:23.184127 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ndvc8_24e58fae-3f60-4a62-ae19-2d3a7396628c/graceful-termination/0.log" Jul 22 16:23:23.188375 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:23.188352 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:23.704747 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:23.704707 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:23.772726 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:23.772689 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:24.601124 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:24.601070 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:24.810875 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:24.810836 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:24.911324 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:24.911228 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:25.011044 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:25.011015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:25.191182 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:25.191078 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:25.291302 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:25.291244 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:25.696662 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:25.696627 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:26.301777 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:26.301740 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:26.301938 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:26.301891 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:23:26.301979 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:26.301949 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls podName:abbf1ead-2639-429c-8a3b-eb8664f3996d nodeName:}" failed. No retries permitted until 2026-07-22 16:23:34.301936045 +0000 UTC m=+57.007378565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qvpmq" (UID: "abbf1ead-2639-429c-8a3b-eb8664f3996d") : secret "insights-runtime-extractor-tls" not found Jul 22 16:23:26.545041 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:26.545005 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:26.688071 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:26.687984 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:26.895382 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:26.895349 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:27.304433 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:27.304392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:27.489353 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:27.489311 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:27.687383 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:27.687290 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:28.288953 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.288916 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:28.518029 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.517992 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:28.518215 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.518107 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:28.520505 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.520476 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46d4b72d-b63d-4e54-a782-abfdd53f3c48-metrics-tls\") pod \"dns-default-g6gmg\" (UID: \"46d4b72d-b63d-4e54-a782-abfdd53f3c48\") " pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:28.520505 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.520505 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"image-registry-694d669fcc-wv78c\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:28.539614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.539556 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:28.550833 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.550802 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:28.679896 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.679862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:28.698765 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.698528 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-6fd7n" podStartSLOduration=7.383943363 podStartE2EDuration="9.698503953s" totalImagesPullingTime="2.31456059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:22.958679714 +0000 UTC m=+45.664122256" watchObservedRunningTime="2026-07-22 16:23:28.698503953 +0000 UTC m=+51.403946497" Jul 22 16:23:28.699841 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.699812 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-694d669fcc-wv78c"] Jul 22 16:23:28.704205 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:28.704171 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc42118_83c7_492f_974d_876c8954257e.slice/crio-746889fab075ceb9d7d793aa7a3ace6eb80a1f419657806cd03dc42c756515d2 WatchSource:0}: Error finding container 746889fab075ceb9d7d793aa7a3ace6eb80a1f419657806cd03dc42c756515d2: Status 404 returned error can't find the container with id 746889fab075ceb9d7d793aa7a3ace6eb80a1f419657806cd03dc42c756515d2 Jul 22 16:23:28.720144 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.720117 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g6gmg"] Jul 22 16:23:28.722868 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:28.722846 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d4b72d_b63d_4e54_a782_abfdd53f3c48.slice/crio-ef6355d0b371bb5cb1b03f2acb4470e00d7649b9df991bbb72181b57b10bf4da WatchSource:0}: Error finding container ef6355d0b371bb5cb1b03f2acb4470e00d7649b9df991bbb72181b57b10bf4da: Status 404 returned error can't find the container with id ef6355d0b371bb5cb1b03f2acb4470e00d7649b9df991bbb72181b57b10bf4da Jul 22 16:23:28.896333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.896298 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:28.956543 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.956506 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g6gmg" event={"ID":"46d4b72d-b63d-4e54-a782-abfdd53f3c48","Type":"ContainerStarted","Data":"ef6355d0b371bb5cb1b03f2acb4470e00d7649b9df991bbb72181b57b10bf4da"} Jul 22 16:23:28.958013 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.957987 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" event={"ID":"bbc42118-83c7-492f-974d-876c8954257e","Type":"ContainerStarted","Data":"c3e0cb4d813cee4be37b38f206b006c2f5f2fb77fc3fa13cb1812af5e12ed94c"} Jul 22 16:23:28.958160 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.958031 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" event={"ID":"bbc42118-83c7-492f-974d-876c8954257e","Type":"ContainerStarted","Data":"746889fab075ceb9d7d793aa7a3ace6eb80a1f419657806cd03dc42c756515d2"} Jul 22 16:23:28.958160 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.958132 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:28.977383 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:28.977335 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" podStartSLOduration=19.977315445 podStartE2EDuration="19.977315445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:28.976021417 +0000 UTC m=+51.681463959" watchObservedRunningTime="2026-07-22 16:23:28.977315445 +0000 UTC m=+51.682757990" Jul 22 16:23:29.084201 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:29.084113 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:29.285343 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:29.285261 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:29.484700 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:29.484583 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:29.584908 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:29.584875 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:29.690522 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:29.690487 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:30.887372 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:30.887330 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:31.081590 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.081551 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:31.256179 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.256142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:31.311145 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.311114 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:31.483758 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.483661 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:31.686483 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.686450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:31.895509 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.895473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:31.968040 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.968009 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g6gmg" event={"ID":"46d4b72d-b63d-4e54-a782-abfdd53f3c48","Type":"ContainerStarted","Data":"c6c0619aa1f9cfae1d3d88e54cf10b9b3ee943d4af27dcf0c10b3a0e1850bcfd"} Jul 22 16:23:31.968245 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.968060 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g6gmg" event={"ID":"46d4b72d-b63d-4e54-a782-abfdd53f3c48","Type":"ContainerStarted","Data":"e77cc34004b3f8e72f7476909350b74376f294afef7e1c1d161e91789dd93d24"} Jul 22 16:23:31.968245 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.968137 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:31.985557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:31.985500 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g6gmg" podStartSLOduration=17.834932641 podStartE2EDuration="19.985480206s" totalImagesPullingTime="2.150547565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:31.983812009 +0000 UTC m=+54.689254542" watchObservedRunningTime="2026-07-22 16:23:31.985480206 +0000 UTC m=+54.690922752" Jul 22 16:23:32.693916 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:32.693883 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:33.082021 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:33.081983 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:33.286062 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:33.286029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:33.493899 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:33.493808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:33.885688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:33.885645 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:34.364745 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:34.364709 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:34.367184 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:34.367155 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abbf1ead-2639-429c-8a3b-eb8664f3996d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qvpmq\" (UID: \"abbf1ead-2639-429c-8a3b-eb8664f3996d\") " pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:34.489577 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:34.489550 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:34.535215 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:34.535174 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qvpmq" Jul 22 16:23:34.657134 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:34.657035 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qvpmq"] Jul 22 16:23:34.659748 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:34.659720 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbf1ead_2639_429c_8a3b_eb8664f3996d.slice/crio-1ea3c248296ab22945cf98220535420397e8a7895dda6321be4d29e5cc58f3e1 WatchSource:0}: Error finding container 1ea3c248296ab22945cf98220535420397e8a7895dda6321be4d29e5cc58f3e1: Status 404 returned error can't find the container with id 1ea3c248296ab22945cf98220535420397e8a7895dda6321be4d29e5cc58f3e1 Jul 22 16:23:34.886697 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:34.886659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:34.981763 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:34.981669 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qvpmq" event={"ID":"abbf1ead-2639-429c-8a3b-eb8664f3996d","Type":"ContainerStarted","Data":"1ea3c248296ab22945cf98220535420397e8a7895dda6321be4d29e5cc58f3e1"} Jul 22 16:23:35.088059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:35.088017 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:35.299873 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:35.299843 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:38.700634 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:38.700597 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:38.760406 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:38.760372 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:38.913584 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:38.913559 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbgk4" Jul 22 16:23:40.000546 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.000506 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qvpmq" event={"ID":"abbf1ead-2639-429c-8a3b-eb8664f3996d","Type":"ContainerStarted","Data":"ea45559ce035492a751fb07893a230f83eacc10ce5afd31eca99ce7b6e809cd7"} Jul 22 16:23:40.908156 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.908111 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm"] Jul 22 16:23:40.911891 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.911866 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7"] Jul 22 16:23:40.912027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.912001 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:40.914867 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.914844 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:40.915523 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.915505 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 16:23:40.915839 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.915815 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 16:23:40.918982 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.918553 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 16:23:40.918982 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.918627 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 16:23:40.918982 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.918664 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 22 16:23:40.918982 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.918852 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 22 16:23:40.919406 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.919386 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 22 16:23:40.919595 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.919579 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 22 16:23:40.931911 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.931886 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm"] Jul 22 16:23:40.939409 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.939180 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7"] Jul 22 16:23:40.977615 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:40.977578 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-694d669fcc-wv78c"] Jul 22 16:23:41.022693 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022665 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-hub\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022722 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022816 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb68c2a4-9218-47eb-8362-05129065d585-klusterlet-config\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvckf\" (UniqueName: \"kubernetes.io/projected/bb68c2a4-9218-47eb-8362-05129065d585-kube-api-access-tvckf\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022882 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27c8256-e890-46ad-975d-1875d1b04c6e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022933 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb68c2a4-9218-47eb-8362-05129065d585-tmp\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022957 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-ca\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.022978 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.023193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.023004 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mp7z\" (UniqueName: \"kubernetes.io/projected/a27c8256-e890-46ad-975d-1875d1b04c6e-kube-api-access-4mp7z\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.024465 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.024438 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c485f5fc9-j6rf8"] Jul 22 16:23:41.027369 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.027347 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.037208 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:23:41.035498 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"image-registry-5c485f5fc9-j6rf8\" is forbidden: User \"system:node:ip-10-0-138-200.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-138-200.ec2.internal' and this object" podUID="77539c43-325f-46a5-bed1-7a78c000a7e7" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.091388 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.091345 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c485f5fc9-j6rf8"] Jul 22 16:23:41.096260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.096226 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:41.123806 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.123775 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb68c2a4-9218-47eb-8362-05129065d585-klusterlet-config\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.124006 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.123831 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-registry-tls\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.124006 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.123863 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77539c43-325f-46a5-bed1-7a78c000a7e7-ca-trust-extracted\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.124006 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.123892 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvckf\" (UniqueName: \"kubernetes.io/projected/bb68c2a4-9218-47eb-8362-05129065d585-kube-api-access-tvckf\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.124006 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.123954 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27c8256-e890-46ad-975d-1875d1b04c6e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.124006 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124001 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb68c2a4-9218-47eb-8362-05129065d585-tmp\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.124223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124028 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-ca\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.124223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124050 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.124223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124082 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mp7z\" (UniqueName: \"kubernetes.io/projected/a27c8256-e890-46ad-975d-1875d1b04c6e-kube-api-access-4mp7z\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.124223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124125 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77539c43-325f-46a5-bed1-7a78c000a7e7-trusted-ca\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.124223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124150 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77539c43-325f-46a5-bed1-7a78c000a7e7-installation-pull-secrets\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.124223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124178 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77539c43-325f-46a5-bed1-7a78c000a7e7-image-registry-private-configuration\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.124223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-hub\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.124504 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124231 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77539c43-325f-46a5-bed1-7a78c000a7e7-registry-certificates\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.124504 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124290 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhdzh\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-kube-api-access-hhdzh\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.124803 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124714 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb68c2a4-9218-47eb-8362-05129065d585-tmp\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.124803 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.124923 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.124787 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-bound-sa-token\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.125628 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.125601 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27c8256-e890-46ad-975d-1875d1b04c6e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.127303 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.127269 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.127431 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.127306 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-hub\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.127693 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.127673 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb68c2a4-9218-47eb-8362-05129065d585-klusterlet-config\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.127772 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.127681 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-ca\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.127830 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.127784 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a27c8256-e890-46ad-975d-1875d1b04c6e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.136438 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.136410 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mp7z\" (UniqueName: \"kubernetes.io/projected/a27c8256-e890-46ad-975d-1875d1b04c6e-kube-api-access-4mp7z\") pod \"cluster-proxy-proxy-agent-5766495ff9-647r7\" (UID: \"a27c8256-e890-46ad-975d-1875d1b04c6e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.136711 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.136678 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvckf\" (UniqueName: \"kubernetes.io/projected/bb68c2a4-9218-47eb-8362-05129065d585-kube-api-access-tvckf\") pod \"klusterlet-addon-workmgr-54fcdcdb88-fldhm\" (UID: \"bb68c2a4-9218-47eb-8362-05129065d585\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.189925 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.189838 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:41.225400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225181 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:41.225400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225333 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77539c43-325f-46a5-bed1-7a78c000a7e7-ca-trust-extracted\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77539c43-325f-46a5-bed1-7a78c000a7e7-trusted-ca\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77539c43-325f-46a5-bed1-7a78c000a7e7-installation-pull-secrets\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77539c43-325f-46a5-bed1-7a78c000a7e7-image-registry-private-configuration\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225478 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77539c43-325f-46a5-bed1-7a78c000a7e7-registry-certificates\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225535 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhdzh\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-kube-api-access-hhdzh\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225566 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-bound-sa-token\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225621 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-registry-tls\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.225970 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.225748 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/77539c43-325f-46a5-bed1-7a78c000a7e7-ca-trust-extracted\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.226467 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.226414 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/77539c43-325f-46a5-bed1-7a78c000a7e7-trusted-ca\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.226899 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.226876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/77539c43-325f-46a5-bed1-7a78c000a7e7-registry-certificates\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.228602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.228580 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-registry-tls\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.229977 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.229691 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/77539c43-325f-46a5-bed1-7a78c000a7e7-image-registry-private-configuration\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.231103 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.231062 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/77539c43-325f-46a5-bed1-7a78c000a7e7-installation-pull-secrets\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.243128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.242015 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" Jul 22 16:23:41.249928 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.249903 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhdzh\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-kube-api-access-hhdzh\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.254185 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.254162 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77539c43-325f-46a5-bed1-7a78c000a7e7-bound-sa-token\") pod \"image-registry-5c485f5fc9-j6rf8\" (UID: \"77539c43-325f-46a5-bed1-7a78c000a7e7\") " pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.338868 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.338558 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:41.392813 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.392782 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm"] Jul 22 16:23:41.394453 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.394366 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:41.398179 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:41.398154 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb68c2a4_9218_47eb_8362_05129065d585.slice/crio-eb414ea23d37e1b2bf06755e1cbc695e7e993f8285e198243133856df1f27f6f WatchSource:0}: Error finding container eb414ea23d37e1b2bf06755e1cbc695e7e993f8285e198243133856df1f27f6f: Status 404 returned error can't find the container with id eb414ea23d37e1b2bf06755e1cbc695e7e993f8285e198243133856df1f27f6f Jul 22 16:23:41.418169 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.417929 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7"] Jul 22 16:23:41.420574 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:41.420537 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda27c8256_e890_46ad_975d_1875d1b04c6e.slice/crio-cb3af9580c556de790f953f0e0474ae8efb52270b4317e1a3e165ca367fb99b8 WatchSource:0}: Error finding container cb3af9580c556de790f953f0e0474ae8efb52270b4317e1a3e165ca367fb99b8: Status 404 returned error can't find the container with id cb3af9580c556de790f953f0e0474ae8efb52270b4317e1a3e165ca367fb99b8 Jul 22 16:23:41.511890 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.511690 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c485f5fc9-j6rf8"] Jul 22 16:23:41.513941 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.513915 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:41.514705 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:41.514673 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77539c43_325f_46a5_bed1_7a78c000a7e7.slice/crio-d87dec4e8ececaf75291e61d213135d74140ae66b369017b5269d86cedf2b2b8 WatchSource:0}: Error finding container d87dec4e8ececaf75291e61d213135d74140ae66b369017b5269d86cedf2b2b8: Status 404 returned error can't find the container with id d87dec4e8ececaf75291e61d213135d74140ae66b369017b5269d86cedf2b2b8 Jul 22 16:23:41.974327 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:41.974294 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g6gmg" Jul 22 16:23:42.009613 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.009582 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" event={"ID":"77539c43-325f-46a5-bed1-7a78c000a7e7","Type":"ContainerStarted","Data":"2890327ef072929c846b1805e07107af2501ecf438171cd5987a32e295c92bcb"} Jul 22 16:23:42.009788 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.009629 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" event={"ID":"77539c43-325f-46a5-bed1-7a78c000a7e7","Type":"ContainerStarted","Data":"d87dec4e8ececaf75291e61d213135d74140ae66b369017b5269d86cedf2b2b8"} Jul 22 16:23:42.009788 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.009694 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:23:42.010993 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.010965 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" event={"ID":"bb68c2a4-9218-47eb-8362-05129065d585","Type":"ContainerStarted","Data":"eb414ea23d37e1b2bf06755e1cbc695e7e993f8285e198243133856df1f27f6f"} Jul 22 16:23:42.012279 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.012254 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" event={"ID":"a27c8256-e890-46ad-975d-1875d1b04c6e","Type":"ContainerStarted","Data":"cb3af9580c556de790f953f0e0474ae8efb52270b4317e1a3e165ca367fb99b8"} Jul 22 16:23:42.073819 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.073766 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" podStartSLOduration=2.073749228 podStartE2EDuration="2.073749228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:42.072922597 +0000 UTC m=+64.778365140" watchObservedRunningTime="2026-07-22 16:23:42.073749228 +0000 UTC m=+64.779191747" Jul 22 16:23:42.260420 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.260320 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:42.376300 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:42.376266 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:43.447604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.447565 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:43.451117 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.450903 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:23:43.477627 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.477594 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df349bb5-3a16-47f3-80dc-4d13678538c2-original-pull-secret\") pod \"global-pull-secret-syncer-ckfjr\" (UID: \"df349bb5-3a16-47f3-80dc-4d13678538c2\") " pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:43.549075 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.549029 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:43.551960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.551931 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:23:43.567847 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.567815 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abe4d75b-3d34-4b30-aabe-7952f9debdb4-metrics-certs\") pod \"network-metrics-daemon-rw9fk\" (UID: \"abe4d75b-3d34-4b30-aabe-7952f9debdb4\") " pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:43.650555 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.650512 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:43.653333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.653304 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vhlm5\"" Jul 22 16:23:43.653594 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.653575 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 16:23:43.661412 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.660961 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rw9fk" Jul 22 16:23:43.664143 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.663704 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ckfjr" Jul 22 16:23:43.664143 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.663926 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 16:23:43.679993 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.679945 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjfxx\" (UniqueName: \"kubernetes.io/projected/507c1878-0b1f-4510-acbf-f5ddfae2ca5c-kube-api-access-jjfxx\") pod \"network-check-target-dczqj\" (UID: \"507c1878-0b1f-4510-acbf-f5ddfae2ca5c\") " pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:43.959181 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.959150 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h7v2l\"" Jul 22 16:23:43.967417 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:43.967377 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:45.035571 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:45.035448 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qvpmq" event={"ID":"abbf1ead-2639-429c-8a3b-eb8664f3996d","Type":"ContainerStarted","Data":"6b49aa3d4133260459d695333dab8918c47bad9479de9b5a514154c206bb9e36"} Jul 22 16:23:45.090768 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:45.090595 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ckfjr"] Jul 22 16:23:45.109240 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:45.109208 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dczqj"] Jul 22 16:23:45.112529 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:45.112500 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507c1878_0b1f_4510_acbf_f5ddfae2ca5c.slice/crio-7ec9db0e2a79aa3df944732ccdf8bc1be8cba5e6f531404cb6e40c3728ba1bff WatchSource:0}: Error finding container 7ec9db0e2a79aa3df944732ccdf8bc1be8cba5e6f531404cb6e40c3728ba1bff: Status 404 returned error can't find the container with id 7ec9db0e2a79aa3df944732ccdf8bc1be8cba5e6f531404cb6e40c3728ba1bff Jul 22 16:23:45.125665 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:45.125499 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rw9fk"] Jul 22 16:23:45.129109 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:23:45.129064 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe4d75b_3d34_4b30_aabe_7952f9debdb4.slice/crio-6731fa0c86ba0ffe666d5af8d179ef667670f86526914ef63c36439d2947aee7 WatchSource:0}: Error finding container 6731fa0c86ba0ffe666d5af8d179ef667670f86526914ef63c36439d2947aee7: Status 404 returned error can't find the container with id 6731fa0c86ba0ffe666d5af8d179ef667670f86526914ef63c36439d2947aee7 Jul 22 16:23:46.040546 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:46.040499 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ckfjr" event={"ID":"df349bb5-3a16-47f3-80dc-4d13678538c2","Type":"ContainerStarted","Data":"8d642e5b1b22331a6643a4cd9b30acd3c7a98889e4a2f46b5403f922f3f13a18"} Jul 22 16:23:46.042052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:46.042020 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rw9fk" event={"ID":"abe4d75b-3d34-4b30-aabe-7952f9debdb4","Type":"ContainerStarted","Data":"6731fa0c86ba0ffe666d5af8d179ef667670f86526914ef63c36439d2947aee7"} Jul 22 16:23:46.043267 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:46.043244 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dczqj" event={"ID":"507c1878-0b1f-4510-acbf-f5ddfae2ca5c","Type":"ContainerStarted","Data":"7ec9db0e2a79aa3df944732ccdf8bc1be8cba5e6f531404cb6e40c3728ba1bff"} Jul 22 16:23:49.817416 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:49.817378 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:49.912121 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:49.911999 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:49.979648 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:49.979606 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:50.155290 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:50.155234 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:50.209023 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:50.208944 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:50.826806 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:50.826768 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:50.983922 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:50.983897 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:23:52.882806 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:52.882770 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:52.985269 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:52.985119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:53.084577 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.083005 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" event={"ID":"bb68c2a4-9218-47eb-8362-05129065d585","Type":"ContainerStarted","Data":"1f33ddb7d2d6b53f798e7002e46de782becb8acb2688790d4af9330ffdd22a25"} Jul 22 16:23:53.084577 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.084222 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:53.085261 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.085220 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" Jul 22 16:23:53.085796 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.085766 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dczqj" event={"ID":"507c1878-0b1f-4510-acbf-f5ddfae2ca5c","Type":"ContainerStarted","Data":"c8ef835da2b9f35586e2ff67cf15575065ab78dabde7c475ddf5bd21f73cbfce"} Jul 22 16:23:53.087686 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.085909 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:23:53.116133 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.115731 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54fcdcdb88-fldhm" podStartSLOduration=4.876910463 podStartE2EDuration="13.115711164s" totalImagesPullingTime="8.238800701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:53.101227348 +0000 UTC m=+75.806669888" watchObservedRunningTime="2026-07-22 16:23:53.115711164 +0000 UTC m=+75.821153710" Jul 22 16:23:53.116275 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.116161 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dczqj" podStartSLOduration=68.321703021 podStartE2EDuration="1m16.116149479s" totalImagesPullingTime="7.794446458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:53.11532665 +0000 UTC m=+75.820769193" watchObservedRunningTime="2026-07-22 16:23:53.116149479 +0000 UTC m=+75.821592024" Jul 22 16:23:53.700712 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.700670 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:53.795813 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.795778 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:53.858489 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.858450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:53.978307 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:53.978204 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:54.052396 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.052363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:54.090887 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.090852 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ckfjr" event={"ID":"df349bb5-3a16-47f3-80dc-4d13678538c2","Type":"ContainerStarted","Data":"c191a362176d199e68db1372ddfd21a7b1ea28cd315fdef642520a6726d70281"} Jul 22 16:23:54.093127 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.093081 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" event={"ID":"a27c8256-e890-46ad-975d-1875d1b04c6e","Type":"ContainerStarted","Data":"09a6e0589d9994f6c1495582d770b65661a5f9f1240a10eb1ac9a6510a267a1a"} Jul 22 16:23:54.093254 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.093133 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" event={"ID":"a27c8256-e890-46ad-975d-1875d1b04c6e","Type":"ContainerStarted","Data":"a0bbabb5a30f2e46a9434165df39026bc8f0e81600ed0a60b298e8b6ab9f13d8"} Jul 22 16:23:54.093254 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.093147 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" event={"ID":"a27c8256-e890-46ad-975d-1875d1b04c6e","Type":"ContainerStarted","Data":"6c74f552f3bc0e6d2463ca65926546328b912ee2daecf6eafd6e8c3b5b87f1e0"} Jul 22 16:23:54.094945 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.094921 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rw9fk" event={"ID":"abe4d75b-3d34-4b30-aabe-7952f9debdb4","Type":"ContainerStarted","Data":"1b2416aa7dc808ed644a91233ba163591146db26efdc329852b50c1b9926c9dd"} Jul 22 16:23:54.095045 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.094968 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rw9fk" event={"ID":"abe4d75b-3d34-4b30-aabe-7952f9debdb4","Type":"ContainerStarted","Data":"b50bd72a87d353da24e09d800e3cfe3e4d7f5e9295a8b27533c437acb6c1ff14"} Jul 22 16:23:54.097035 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.097011 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qvpmq" event={"ID":"abbf1ead-2639-429c-8a3b-eb8664f3996d","Type":"ContainerStarted","Data":"756e6dd7092107ff65212195bcfa504e289b8f040b2eb3a4c842250b3205c416"} Jul 22 16:23:54.108785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.108742 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ckfjr" podStartSLOduration=69.250874689 podStartE2EDuration="1m17.10872722s" totalImagesPullingTime="7.857852531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:54.108356805 +0000 UTC m=+76.813799350" watchObservedRunningTime="2026-07-22 16:23:54.10872722 +0000 UTC m=+76.814169759" Jul 22 16:23:54.124160 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.124109 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rw9fk" podStartSLOduration=69.35436 podStartE2EDuration="1m17.124070556s" totalImagesPullingTime="7.769710556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:54.123769208 +0000 UTC m=+76.829211750" watchObservedRunningTime="2026-07-22 16:23:54.124070556 +0000 UTC m=+76.829513098" Jul 22 16:23:54.141787 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.141733 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5766495ff9-647r7" podStartSLOduration=2.670714864 podStartE2EDuration="14.141717753s" totalImagesPullingTime="11.471002889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:54.141281204 +0000 UTC m=+76.846723746" watchObservedRunningTime="2026-07-22 16:23:54.141717753 +0000 UTC m=+76.847160293" Jul 22 16:23:54.159138 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.159059 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qvpmq" podStartSLOduration=18.044143037 podStartE2EDuration="36.159038724s" totalImagesPullingTime="18.114895687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:23:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:23:54.15883551 +0000 UTC m=+76.864278054" watchObservedRunningTime="2026-07-22 16:23:54.159038724 +0000 UTC m=+76.864481267" Jul 22 16:23:54.222329 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.222295 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:54.323440 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.323401 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:54.426319 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:54.426284 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:58.051845 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:58.051802 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:58.152323 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:58.152280 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:58.255485 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:58.255439 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:59.875059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:59.875025 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:23:59.976024 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:23:59.975990 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.101013 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.100977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.203187 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.203080 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.323749 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.323710 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.429264 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.429225 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q2ctm"] Jul 22 16:24:00.433872 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.433849 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.437062 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.437028 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 16:24:00.437062 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.437029 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jsxbv\"" Jul 22 16:24:00.437266 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.437030 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 16:24:00.437407 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.437392 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 16:24:00.437991 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.437970 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 16:24:00.438135 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.438015 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 16:24:00.438300 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.438268 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 16:24:00.466536 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.466449 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.564535 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.564495 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.587504 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587472 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9508b4dc-16a7-4705-b35a-c85e0df8202d-metrics-client-ca\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79j47\" (UniqueName: \"kubernetes.io/projected/9508b4dc-16a7-4705-b35a-c85e0df8202d-kube-api-access-79j47\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587540 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587566 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-wtmp\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587591 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587635 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-tls\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587656 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-textfile\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587698 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-sys\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.587729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.587716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-root\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.667106 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.666868 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689030 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-tls\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689082 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-textfile\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689148 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-sys\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689174 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-root\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9508b4dc-16a7-4705-b35a-c85e0df8202d-metrics-client-ca\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689238 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79j47\" (UniqueName: \"kubernetes.io/projected/9508b4dc-16a7-4705-b35a-c85e0df8202d-kube-api-access-79j47\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689274 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689306 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-wtmp\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689333 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.690107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.689836 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-root\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.691353 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.691329 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-sys\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.691553 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.691373 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-textfile\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.691553 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.691556 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9508b4dc-16a7-4705-b35a-c85e0df8202d-metrics-client-ca\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.691717 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.691417 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-wtmp\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.691770 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.691753 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.692663 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.692635 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.693481 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.693451 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9508b4dc-16a7-4705-b35a-c85e0df8202d-node-exporter-tls\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.699247 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.699226 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79j47\" (UniqueName: \"kubernetes.io/projected/9508b4dc-16a7-4705-b35a-c85e0df8202d-kube-api-access-79j47\") pod \"node-exporter-q2ctm\" (UID: \"9508b4dc-16a7-4705-b35a-c85e0df8202d\") " pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.743387 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.743304 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:00.745191 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.745164 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q2ctm" Jul 22 16:24:00.757683 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:24:00.757658 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9508b4dc_16a7_4705_b35a_c85e0df8202d.slice/crio-44c2f4f6002ce7e497a1b5ffba7ff11ae04ba20f09d2a1ec4d29de1b2ae9530c WatchSource:0}: Error finding container 44c2f4f6002ce7e497a1b5ffba7ff11ae04ba20f09d2a1ec4d29de1b2ae9530c: Status 404 returned error can't find the container with id 44c2f4f6002ce7e497a1b5ffba7ff11ae04ba20f09d2a1ec4d29de1b2ae9530c Jul 22 16:24:00.981557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:00.981518 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:01.121966 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:01.121925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2ctm" event={"ID":"9508b4dc-16a7-4705-b35a-c85e0df8202d","Type":"ContainerStarted","Data":"44c2f4f6002ce7e497a1b5ffba7ff11ae04ba20f09d2a1ec4d29de1b2ae9530c"} Jul 22 16:24:01.371106 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:01.371043 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:01.459485 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:01.459385 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:01.681217 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:01.681182 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:01.992963 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:01.992929 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:02.073867 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:02.073780 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:02.126432 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:02.126403 2555 generic.go:356] "Generic (PLEG): container finished" podID="9508b4dc-16a7-4705-b35a-c85e0df8202d" containerID="29720332a68b1158112b3310b0032680869b9492db06b646cc9e6b86bcc4c126" exitCode=0 Jul 22 16:24:02.126586 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:02.126439 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2ctm" event={"ID":"9508b4dc-16a7-4705-b35a-c85e0df8202d","Type":"ContainerDied","Data":"29720332a68b1158112b3310b0032680869b9492db06b646cc9e6b86bcc4c126"} Jul 22 16:24:02.139846 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:02.139802 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:03.025286 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.025261 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c485f5fc9-j6rf8" Jul 22 16:24:03.131789 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.131760 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2ctm" event={"ID":"9508b4dc-16a7-4705-b35a-c85e0df8202d","Type":"ContainerStarted","Data":"37f0b7484d0f4831c4f3084e50a865c2a79de2601f2182efa7d67c24de2ca32c"} Jul 22 16:24:03.131940 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.131808 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2ctm" event={"ID":"9508b4dc-16a7-4705-b35a-c85e0df8202d","Type":"ContainerStarted","Data":"1cc7faba70fd6980256fd8ee5d680072fb07effd008ca8c036f3388e140ac48e"} Jul 22 16:24:03.148539 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.148497 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q2ctm" podStartSLOduration=2.050942442 podStartE2EDuration="3.148482116s" totalImagesPullingTime="1.027321049s" totalInitContainerRuntime="70.218625ms" isStatefulPod=true podCreationTimestamp="2026-07-22 16:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:24:03.148433717 +0000 UTC m=+85.853876258" watchObservedRunningTime="2026-07-22 16:24:03.148482116 +0000 UTC m=+85.853924658" Jul 22 16:24:03.163420 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.163388 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:03.262980 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.262942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:03.363301 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.363257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:03.466294 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.466252 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:03.523245 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:03.523196 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:04.036244 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:04.036211 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:05.119381 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:05.119342 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:05.377369 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:05.377252 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:06.003004 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.002973 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" podUID="bbc42118-83c7-492f-974d-876c8954257e" containerName="registry" containerID="cri-o://c3e0cb4d813cee4be37b38f206b006c2f5f2fb77fc3fa13cb1812af5e12ed94c" gracePeriod=30 Jul 22 16:24:06.142954 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.142920 2555 generic.go:356] "Generic (PLEG): container finished" podID="bbc42118-83c7-492f-974d-876c8954257e" containerID="c3e0cb4d813cee4be37b38f206b006c2f5f2fb77fc3fa13cb1812af5e12ed94c" exitCode=0 Jul 22 16:24:06.143332 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.142999 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" event={"ID":"bbc42118-83c7-492f-974d-876c8954257e","Type":"ContainerDied","Data":"c3e0cb4d813cee4be37b38f206b006c2f5f2fb77fc3fa13cb1812af5e12ed94c"} Jul 22 16:24:06.250104 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.250067 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:24:06.436190 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436163 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-trusted-ca\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436200 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-image-registry-private-configuration\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436224 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436244 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxkhj\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-kube-api-access-kxkhj\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436270 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-bound-sa-token\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436315 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbc42118-83c7-492f-974d-876c8954257e-ca-trust-extracted\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436626 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436527 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-installation-pull-secrets\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436626 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436590 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-registry-certificates\") pod \"bbc42118-83c7-492f-974d-876c8954257e\" (UID: \"bbc42118-83c7-492f-974d-876c8954257e\") " Jul 22 16:24:06.436626 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436596 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:24:06.437157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.436927 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-trusted-ca\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.437595 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.437566 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:24:06.439046 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.439021 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:24:06.439175 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.439044 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:24:06.439175 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.439144 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:24:06.439175 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.439165 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:24:06.439283 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.439236 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-kube-api-access-kxkhj" (OuterVolumeSpecName: "kube-api-access-kxkhj") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "kube-api-access-kxkhj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:24:06.445576 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.445554 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc42118-83c7-492f-974d-876c8954257e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bbc42118-83c7-492f-974d-876c8954257e" (UID: "bbc42118-83c7-492f-974d-876c8954257e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:24:06.537729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.537692 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-image-registry-private-configuration\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.537729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.537722 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-registry-tls\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.537729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.537734 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxkhj\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-kube-api-access-kxkhj\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.537958 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.537744 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbc42118-83c7-492f-974d-876c8954257e-bound-sa-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.537958 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.537753 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbc42118-83c7-492f-974d-876c8954257e-ca-trust-extracted\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.537958 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.537761 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbc42118-83c7-492f-974d-876c8954257e-installation-pull-secrets\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.537958 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.537768 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbc42118-83c7-492f-974d-876c8954257e-registry-certificates\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:24:06.671749 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.671714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:06.754839 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.754758 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:06.918672 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:06.918640 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:07.038364 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.038332 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:07.148007 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.147982 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" Jul 22 16:24:07.148460 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.147982 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-694d669fcc-wv78c" event={"ID":"bbc42118-83c7-492f-974d-876c8954257e","Type":"ContainerDied","Data":"746889fab075ceb9d7d793aa7a3ace6eb80a1f419657806cd03dc42c756515d2"} Jul 22 16:24:07.148460 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.148133 2555 scope.go:117] "RemoveContainer" containerID="c3e0cb4d813cee4be37b38f206b006c2f5f2fb77fc3fa13cb1812af5e12ed94c" Jul 22 16:24:07.179008 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.178986 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-694d669fcc-wv78c"] Jul 22 16:24:07.187883 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.187864 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-694d669fcc-wv78c"] Jul 22 16:24:07.232853 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.232819 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:07.607705 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.607667 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:07.746420 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.746381 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbc42118-83c7-492f-974d-876c8954257e" path="/var/lib/kubelet/pods/bbc42118-83c7-492f-974d-876c8954257e/volumes" Jul 22 16:24:07.832152 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:07.832125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:08.044341 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:08.044279 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:08.148899 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:08.148857 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:08.247774 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:08.247735 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:08.416030 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:08.415939 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:08.737168 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:08.737071 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:08.837596 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:08.837552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:08.898559 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:08.898531 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:09.439780 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:09.439731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:09.497390 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:09.497356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:11.106504 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:11.106463 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:11.175051 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:11.175023 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:13.114160 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:13.114122 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:13.215396 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:13.215363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:13.315438 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:13.315405 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:22.833425 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:22.833385 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:22.885630 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:22.885596 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:23.698890 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:23.698850 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:23.749863 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:23.749830 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:24.099726 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:24.099699 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dczqj" Jul 22 16:24:27.048178 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:27.048134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:33.245414 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:33.245367 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:33.330556 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:33.330525 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:35.042724 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:35.042691 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:35.092397 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:35.092352 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:37.058656 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:37.058608 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:37.114623 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:37.114574 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:38.038257 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:38.038219 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:38.119845 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:38.119823 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2ctm_9508b4dc-16a7-4705-b35a-c85e0df8202d/init-textfile/0.log" Jul 22 16:24:38.321260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:38.321177 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2ctm_9508b4dc-16a7-4705-b35a-c85e0df8202d/node-exporter/0.log" Jul 22 16:24:38.520112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:38.520061 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2ctm_9508b4dc-16a7-4705-b35a-c85e0df8202d/kube-rbac-proxy/0.log" Jul 22 16:24:38.695208 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:38.695123 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:38.795363 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:38.795331 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:44.686157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:44.686121 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:44.789738 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:44.789703 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:44.886409 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:44.886382 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:45.518950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:45.518908 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7mrnv_0618f128-da4f-48f9-80f8-2556651f28e4/serve-healthcheck-canary/0.log" Jul 22 16:24:53.700753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:53.700706 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:24:53.763566 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:24:53.763526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:16.471434 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:16.471393 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:16.531197 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:16.531161 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:21.558975 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:21.558934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:28.322824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:28.322787 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:28.381977 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:28.381922 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:32.606305 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:32.606265 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:32.659473 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:32.659444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:32.717400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:32.717359 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:38.705698 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:38.705655 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:38.807268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:38.807236 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:25:57.065557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:25:57.065509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:26:04.184663 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:26:04.184623 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:26:04.236964 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:26:04.236929 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:26:18.178579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:26:18.178539 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:26:41.685465 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:26:41.685429 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:26:42.508562 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:26:42.508526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:26:42.602281 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:26:42.602246 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:02.047292 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:02.047204 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:02.100923 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:02.100895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:09.290705 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:09.290668 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:10.162824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:10.162775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:12.910645 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:12.910604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:13.010490 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:13.010456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:13.444607 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:13.444568 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:13.840201 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:13.840158 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:13.977803 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:13.977766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:14.447877 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:14.447847 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:14.618018 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:14.617987 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:16.236362 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:16.236327 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:16.635381 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:16.635347 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:19.686849 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:19.686817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:19.760894 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:19.760854 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:23.635062 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:23.635021 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:30.109600 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:30.109533 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:30.206968 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:30.206934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:35.736679 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:35.736640 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:35.800936 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:35.800898 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:37.613778 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:37.613739 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 16:27:37.887358 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:37.887264 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:47.758855 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:47.758814 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:47.825171 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:47.825135 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:52.142391 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.142353 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:52.208319 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.208283 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr"] Jul 22 16:27:52.208533 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.208521 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc42118-83c7-492f-974d-876c8954257e" containerName="registry" Jul 22 16:27:52.208587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.208535 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc42118-83c7-492f-974d-876c8954257e" containerName="registry" Jul 22 16:27:52.208587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.208573 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc42118-83c7-492f-974d-876c8954257e" containerName="registry" Jul 22 16:27:52.211301 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.211283 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.215444 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.215425 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wdh7w\"" Jul 22 16:27:52.216661 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.216598 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 16:27:52.216661 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.216599 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 16:27:52.216846 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.216775 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 16:27:52.216948 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.216930 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 16:27:52.217120 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.217080 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:27:52.217821 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.217521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9jvc\" (UniqueName: \"kubernetes.io/projected/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-kube-api-access-q9jvc\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.217821 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.217566 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-metrics-cert\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.217821 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.217599 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-cert\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.217821 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.217693 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-manager-config\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.252639 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.252605 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr"] Jul 22 16:27:52.273564 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.273529 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:52.318365 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.318331 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-manager-config\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.318365 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.318366 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9jvc\" (UniqueName: \"kubernetes.io/projected/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-kube-api-access-q9jvc\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.318583 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.318394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-metrics-cert\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.318583 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.318415 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-cert\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.319079 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.319052 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-manager-config\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.321148 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.321126 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-metrics-cert\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.321254 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.321160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-cert\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.344614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.344579 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9jvc\" (UniqueName: \"kubernetes.io/projected/7c4a9d28-c931-441f-8fcf-a5caf5607b5d-kube-api-access-q9jvc\") pod \"lws-controller-manager-65dd5d6b6d-b68tr\" (UID: \"7c4a9d28-c931-441f-8fcf-a5caf5607b5d\") " pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.408176 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.408066 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:52.520632 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.520599 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:52.707442 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:27:52.707405 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c4a9d28_c931_441f_8fcf_a5caf5607b5d.slice/crio-0799bf5b4a16e2506817b829ac9c7a14df9315d914b5602a9af971551ec68dd4 WatchSource:0}: Error finding container 0799bf5b4a16e2506817b829ac9c7a14df9315d914b5602a9af971551ec68dd4: Status 404 returned error can't find the container with id 0799bf5b4a16e2506817b829ac9c7a14df9315d914b5602a9af971551ec68dd4 Jul 22 16:27:52.709609 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.709589 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr"] Jul 22 16:27:52.709791 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.709774 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:27:52.745128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.745072 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:52.887874 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:52.887839 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" event={"ID":"7c4a9d28-c931-441f-8fcf-a5caf5607b5d","Type":"ContainerStarted","Data":"0799bf5b4a16e2506817b829ac9c7a14df9315d914b5602a9af971551ec68dd4"} Jul 22 16:27:53.290283 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.290240 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:53.458406 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.458369 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:53.493360 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.493324 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd"] Jul 22 16:27:53.496838 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.496775 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.499784 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.499725 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 16:27:53.499912 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.499829 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pf667\"" Jul 22 16:27:53.499972 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.499957 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 16:27:53.500027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.499985 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 16:27:53.500027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.500008 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 16:27:53.519460 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.519432 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd"] Jul 22 16:27:53.525434 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.525411 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25a6dfc5-f101-4f84-a7b3-20e7c786e615-tmp\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.525543 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.525454 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/25a6dfc5-f101-4f84-a7b3-20e7c786e615-manifests\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.525543 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.525477 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwntk\" (UniqueName: \"kubernetes.io/projected/25a6dfc5-f101-4f84-a7b3-20e7c786e615-kube-api-access-qwntk\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.525543 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.525534 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/25a6dfc5-f101-4f84-a7b3-20e7c786e615-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.525678 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.525571 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25a6dfc5-f101-4f84-a7b3-20e7c786e615-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.626640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.626151 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/25a6dfc5-f101-4f84-a7b3-20e7c786e615-manifests\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.626640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.626203 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwntk\" (UniqueName: \"kubernetes.io/projected/25a6dfc5-f101-4f84-a7b3-20e7c786e615-kube-api-access-qwntk\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.626640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.626240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/25a6dfc5-f101-4f84-a7b3-20e7c786e615-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.626640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.626290 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25a6dfc5-f101-4f84-a7b3-20e7c786e615-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.626640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.626345 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25a6dfc5-f101-4f84-a7b3-20e7c786e615-tmp\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.626999 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.626798 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/25a6dfc5-f101-4f84-a7b3-20e7c786e615-manifests\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.629599 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.629570 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/25a6dfc5-f101-4f84-a7b3-20e7c786e615-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.629728 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.629679 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/25a6dfc5-f101-4f84-a7b3-20e7c786e615-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.629994 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.629932 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25a6dfc5-f101-4f84-a7b3-20e7c786e615-tmp\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.636257 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.636227 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwntk\" (UniqueName: \"kubernetes.io/projected/25a6dfc5-f101-4f84-a7b3-20e7c786e615-kube-api-access-qwntk\") pod \"opendatahub-operator-controller-manager-9b566c8c6-wcbdd\" (UID: \"25a6dfc5-f101-4f84-a7b3-20e7c786e615\") " pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.661696 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.661659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:53.767809 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.767745 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:53.810422 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.810388 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:53.862839 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.862804 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:53.961895 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.961848 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:53.980806 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:53.980629 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd"] Jul 22 16:27:53.983875 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:27:53.983853 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a6dfc5_f101_4f84_a7b3_20e7c786e615.slice/crio-5d905820a35a51d426132ac368427bb0cdcffdcda16a726fb8a279864c4e4408 WatchSource:0}: Error finding container 5d905820a35a51d426132ac368427bb0cdcffdcda16a726fb8a279864c4e4408: Status 404 returned error can't find the container with id 5d905820a35a51d426132ac368427bb0cdcffdcda16a726fb8a279864c4e4408 Jul 22 16:27:54.115681 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:54.115605 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:54.196740 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:54.196632 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:54.897950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:54.897913 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" event={"ID":"25a6dfc5-f101-4f84-a7b3-20e7c786e615","Type":"ContainerStarted","Data":"5d905820a35a51d426132ac368427bb0cdcffdcda16a726fb8a279864c4e4408"} Jul 22 16:27:55.904927 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:55.904887 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" event={"ID":"7c4a9d28-c931-441f-8fcf-a5caf5607b5d","Type":"ContainerStarted","Data":"784f1860660da99609be19f3c4b572e148337aa0dd596d7140d3ecd1ec9b5ecf"} Jul 22 16:27:55.905452 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:55.905167 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:27:55.929291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:55.929232 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" podStartSLOduration=1.417784651 podStartE2EDuration="3.929208083s" totalImagesPullingTime="2.511423432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:27:55.927947484 +0000 UTC m=+318.633390038" watchObservedRunningTime="2026-07-22 16:27:55.929208083 +0000 UTC m=+318.634650625" Jul 22 16:27:56.493835 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:56.493781 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:56.596782 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:56.596747 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:57.914065 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:57.913975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" event={"ID":"25a6dfc5-f101-4f84-a7b3-20e7c786e615","Type":"ContainerStarted","Data":"44e39a47fe8aa9f3054001bb1829f810b25c2bf6e66af57ab22f04ea0c4a41dc"} Jul 22 16:27:58.343380 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:58.343346 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:58.442991 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:58.442958 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:27:58.919022 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:58.918985 2555 generic.go:356] "Generic (PLEG): container finished" podID="25a6dfc5-f101-4f84-a7b3-20e7c786e615" containerID="44e39a47fe8aa9f3054001bb1829f810b25c2bf6e66af57ab22f04ea0c4a41dc" exitCode=0 Jul 22 16:27:58.919451 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:58.919077 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" event={"ID":"25a6dfc5-f101-4f84-a7b3-20e7c786e615","Type":"ContainerDied","Data":"44e39a47fe8aa9f3054001bb1829f810b25c2bf6e66af57ab22f04ea0c4a41dc"} Jul 22 16:27:59.924259 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:59.924225 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" event={"ID":"25a6dfc5-f101-4f84-a7b3-20e7c786e615","Type":"ContainerStarted","Data":"6b471e7581e145cb63586512281bd2f7dab03ca4f8b77ff9280f13250124bf6d"} Jul 22 16:27:59.924677 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:59.924331 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:27:59.948412 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:27:59.948356 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" podStartSLOduration=1.914141126 podStartE2EDuration="6.948340494s" totalImagesPullingTime="4.03559485s" totalInitContainerRuntime="998.604518ms" isStatefulPod=false podCreationTimestamp="2026-07-22 16:27:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:27:59.945776817 +0000 UTC m=+322.651219360" watchObservedRunningTime="2026-07-22 16:27:59.948340494 +0000 UTC m=+322.653783036" Jul 22 16:28:06.910516 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:06.910490 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65dd5d6b6d-b68tr" Jul 22 16:28:10.929973 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:10.929946 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9b566c8c6-wcbdd" Jul 22 16:28:12.494783 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:12.494747 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:12.613044 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:12.613013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:12.949052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:12.949012 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:13.215441 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:13.215351 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:13.402809 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:13.402775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:13.540209 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:13.540167 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:13.632660 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:13.632621 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:14.294805 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:14.294765 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:14.515690 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:14.515655 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:14.772675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:14.772591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:14.872272 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:14.872237 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:16.368213 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:16.368171 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:21.087294 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.087262 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:21.171672 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.171638 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:21.871717 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.871690 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj"] Jul 22 16:28:21.886212 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.886187 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.891827 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.891802 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 16:28:21.892038 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.892015 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4r56h\"" Jul 22 16:28:21.893910 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.893240 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 16:28:21.893910 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.893672 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 16:28:21.894555 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.894535 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj"] Jul 22 16:28:21.920374 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.920333 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:21.949126 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949075 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949142 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949192 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca0de617-2373-42f4-928e-981572640e95-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949387 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949263 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949387 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949290 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jg7s\" (UniqueName: \"kubernetes.io/projected/ca0de617-2373-42f4-928e-981572640e95-kube-api-access-7jg7s\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949387 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949342 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0de617-2373-42f4-928e-981572640e95-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949387 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949363 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:21.949387 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:21.949379 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0de617-2373-42f4-928e-981572640e95-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050559 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050559 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050563 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050580 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca0de617-2373-42f4-928e-981572640e95-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050625 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050655 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jg7s\" (UniqueName: \"kubernetes.io/projected/ca0de617-2373-42f4-928e-981572640e95-kube-api-access-7jg7s\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050711 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0de617-2373-42f4-928e-981572640e95-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050733 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050756 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0de617-2373-42f4-928e-981572640e95-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.050785 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.050780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.051130 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.051003 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.051201 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.051155 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.051248 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.051214 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.051281 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.051261 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.051414 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.051394 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca0de617-2373-42f4-928e-981572640e95-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.053130 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.053111 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca0de617-2373-42f4-928e-981572640e95-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.053206 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.053160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca0de617-2373-42f4-928e-981572640e95-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.066849 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.066819 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jg7s\" (UniqueName: \"kubernetes.io/projected/ca0de617-2373-42f4-928e-981572640e95-kube-api-access-7jg7s\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.071205 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.071173 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca0de617-2373-42f4-928e-981572640e95-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vgfnj\" (UID: \"ca0de617-2373-42f4-928e-981572640e95\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.146634 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.146525 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:22.197161 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.197122 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:22.296735 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.296699 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:22.336081 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.336015 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj"] Jul 22 16:28:22.338267 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:28:22.338233 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca0de617_2373_42f4_928e_981572640e95.slice/crio-715996ede733312b08100bfd120d0cca2b8fddc728df6cba46d8370f8f939b6f WatchSource:0}: Error finding container 715996ede733312b08100bfd120d0cca2b8fddc728df6cba46d8370f8f939b6f: Status 404 returned error can't find the container with id 715996ede733312b08100bfd120d0cca2b8fddc728df6cba46d8370f8f939b6f Jul 22 16:28:22.695146 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.695112 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx"] Jul 22 16:28:22.699648 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.699629 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.702333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.702310 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 16:28:22.702459 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.702345 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 16:28:22.702459 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.702443 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mjcn6\"" Jul 22 16:28:22.702659 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.702643 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 16:28:22.722713 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.722694 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx"] Jul 22 16:28:22.857417 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.857378 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc7dr\" (UniqueName: \"kubernetes.io/projected/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-kube-api-access-gc7dr\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.857602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.857435 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.857602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.857457 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.857602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.857495 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.857602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.857515 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.857602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.857537 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.857602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.857554 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.958484 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.958394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.958484 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.958437 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.958484 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.958458 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.958484 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.958480 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.958812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.958500 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.958812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.958545 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc7dr\" (UniqueName: \"kubernetes.io/projected/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-kube-api-access-gc7dr\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.958812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.958608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.959282 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.959259 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.960958 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.960934 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.961053 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.960972 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.961361 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.961342 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.961402 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.961360 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.973970 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.973945 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:22.975887 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:22.975866 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc7dr\" (UniqueName: \"kubernetes.io/projected/50046a88-7376-4b4b-a4b3-fdb5f1ca989f-kube-api-access-gc7dr\") pod \"istiod-openshift-gateway-94bb8fbfd-94nwx\" (UID: \"50046a88-7376-4b4b-a4b3-fdb5f1ca989f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:23.004636 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:23.004599 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" event={"ID":"ca0de617-2373-42f4-928e-981572640e95","Type":"ContainerStarted","Data":"715996ede733312b08100bfd120d0cca2b8fddc728df6cba46d8370f8f939b6f"} Jul 22 16:28:23.008800 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:23.008783 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:23.156781 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:23.156746 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx"] Jul 22 16:28:23.159757 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:28:23.159729 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50046a88_7376_4b4b_a4b3_fdb5f1ca989f.slice/crio-cb114379dafea5f19c4f65739f0c06307c026f4c33382142257e294fee96ea20 WatchSource:0}: Error finding container cb114379dafea5f19c4f65739f0c06307c026f4c33382142257e294fee96ea20: Status 404 returned error can't find the container with id cb114379dafea5f19c4f65739f0c06307c026f4c33382142257e294fee96ea20 Jul 22 16:28:24.011323 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:24.011274 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" event={"ID":"50046a88-7376-4b4b-a4b3-fdb5f1ca989f","Type":"ContainerStarted","Data":"cb114379dafea5f19c4f65739f0c06307c026f4c33382142257e294fee96ea20"} Jul 22 16:28:25.377808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:25.377686 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:25.498641 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:25.498524 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:25.649618 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:25.649510 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:25.821672 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:25.821598 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:26.302704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.302658 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:28:26.302815 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.302732 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:28:26.302815 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.302766 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:28:26.304232 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.304193 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:28:26.304316 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.304285 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:28:26.678059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.677938 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:26.799294 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.799254 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:26.908078 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:26.908035 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:28:27.024906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.024795 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" event={"ID":"ca0de617-2373-42f4-928e-981572640e95","Type":"ContainerStarted","Data":"5051f9e73a55e2d30ab36879f49764cfa2c0b8c7784e1101f3f5b9de0b639a68"} Jul 22 16:28:27.026542 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.026518 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" event={"ID":"50046a88-7376-4b4b-a4b3-fdb5f1ca989f","Type":"ContainerStarted","Data":"fd4fff3b036f1c2431400de1ca6e5a893e7ad23a5b4ff452a9ac26d967b309ba"} Jul 22 16:28:27.026736 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.026717 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:27.028251 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.028229 2555 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-94nwx container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 16:28:27.028338 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.028275 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" podUID="50046a88-7376-4b4b-a4b3-fdb5f1ca989f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 16:28:27.045501 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.045456 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" podStartSLOduration=2.083149337 podStartE2EDuration="6.045442242s" totalImagesPullingTime="3.962292905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:28:27.044734109 +0000 UTC m=+349.750176654" watchObservedRunningTime="2026-07-22 16:28:27.045442242 +0000 UTC m=+349.750884975" Jul 22 16:28:27.197597 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.197559 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:27.198999 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.198963 2555 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused" start-of-body= Jul 22 16:28:27.199158 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:27.199023 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" podUID="ca0de617-2373-42f4-928e-981572640e95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused" Jul 22 16:28:28.030849 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:28.030822 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" Jul 22 16:28:28.055906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:28.055858 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-94nwx" podStartSLOduration=2.913838678 podStartE2EDuration="6.055840793s" totalImagesPullingTime="3.142002115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:28:27.071636421 +0000 UTC m=+349.777078966" watchObservedRunningTime="2026-07-22 16:28:28.055840793 +0000 UTC m=+350.761283335" Jul 22 16:28:28.201555 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:28.201519 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:29.033877 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:29.033844 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:28:29.034787 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:28:29.034769 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vgfnj" Jul 22 16:29:14.090399 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:14.090357 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:14.159906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:14.159863 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:14.337716 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:14.337678 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:20.108323 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:20.108291 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:20.164854 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:20.164805 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:20.250145 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:20.250076 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:20.301965 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:20.301932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:20.852794 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:20.852761 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:30.326297 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:30.326257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:30.377723 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:30.377679 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:30.782802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:30.782773 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:30.882959 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:30.882925 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:30.942076 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:30.942034 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:37.552931 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:37.552881 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:39.829284 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:39.829251 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:40.954427 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:40.954391 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:41.062213 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:41.062177 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:46.764039 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:46.763992 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:46.982608 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:46.982571 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:49.342494 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:49.342453 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:49.594811 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:49.594722 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:49.747495 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:49.747456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:52.030113 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:52.030052 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:53.863855 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:53.863822 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:53.967265 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:53.967221 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:54.073257 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:54.073217 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:54.279659 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:54.279626 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:57.459246 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:57.459209 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:29:57.555239 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:29:57.555206 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:03.657404 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:03.657321 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:03.773530 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:03.773481 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:06.746144 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:06.746111 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:10.044400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:10.044366 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:10.145497 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:10.145456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:12.625476 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:12.625443 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:12.728238 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:12.728200 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:15.412108 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:15.412060 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:15.511964 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:15.511926 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:22.560378 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:22.560341 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:22.687333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:22.687305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:29.057367 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:29.057325 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:29.159270 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:29.159230 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:45.461257 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:45.461221 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:45.870876 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:45.870839 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:51.232473 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.232434 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:51.327542 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.327511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:51.428038 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.427987 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:51.568306 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.568274 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:30:51.571286 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.571268 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:51.573873 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.573851 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 16:30:51.573966 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.573853 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 16:30:51.574045 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.574022 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l2vmv\"" Jul 22 16:30:51.575010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.574997 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 16:30:51.581388 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.581368 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:30:51.611778 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.611745 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:30:51.744272 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.744237 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gdlm\" (UniqueName: \"kubernetes.io/projected/e7aa9cf6-5081-4c71-be7c-1dbae4c303c6-kube-api-access-4gdlm\") pod \"limitador-limitador-54d6d988d5-xvsmd\" (UID: \"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:51.744442 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.744296 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7aa9cf6-5081-4c71-be7c-1dbae4c303c6-config-file\") pod \"limitador-limitador-54d6d988d5-xvsmd\" (UID: \"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:51.845689 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.845597 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gdlm\" (UniqueName: \"kubernetes.io/projected/e7aa9cf6-5081-4c71-be7c-1dbae4c303c6-kube-api-access-4gdlm\") pod \"limitador-limitador-54d6d988d5-xvsmd\" (UID: \"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:51.845689 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.845652 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7aa9cf6-5081-4c71-be7c-1dbae4c303c6-config-file\") pod \"limitador-limitador-54d6d988d5-xvsmd\" (UID: \"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:51.846322 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.846303 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7aa9cf6-5081-4c71-be7c-1dbae4c303c6-config-file\") pod \"limitador-limitador-54d6d988d5-xvsmd\" (UID: \"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:51.854307 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.854284 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gdlm\" (UniqueName: \"kubernetes.io/projected/e7aa9cf6-5081-4c71-be7c-1dbae4c303c6-kube-api-access-4gdlm\") pod \"limitador-limitador-54d6d988d5-xvsmd\" (UID: \"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:51.882173 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:51.882147 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:52.015944 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:52.015912 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:30:52.018662 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:30:52.018636 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7aa9cf6_5081_4c71_be7c_1dbae4c303c6.slice/crio-59b7e890dee502fb89eace4d1252ca3eeb1aa6f39661a22bc07be33666168677 WatchSource:0}: Error finding container 59b7e890dee502fb89eace4d1252ca3eeb1aa6f39661a22bc07be33666168677: Status 404 returned error can't find the container with id 59b7e890dee502fb89eace4d1252ca3eeb1aa6f39661a22bc07be33666168677 Jul 22 16:30:52.047538 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:52.047480 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:52.107400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:52.107315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:30:52.554198 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:52.554146 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" event={"ID":"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6","Type":"ContainerStarted","Data":"59b7e890dee502fb89eace4d1252ca3eeb1aa6f39661a22bc07be33666168677"} Jul 22 16:30:54.564263 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:54.564226 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" event={"ID":"e7aa9cf6-5081-4c71-be7c-1dbae4c303c6","Type":"ContainerStarted","Data":"ef86759929600e5e49907b4439c7c5fe94f4a9a1ccdd369a136958e38cab243b"} Jul 22 16:30:54.564643 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:54.564304 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:30:54.581105 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:30:54.581046 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" podStartSLOduration=1.206543136 podStartE2EDuration="3.581008016s" totalImagesPullingTime="2.37446488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:30:54.580058078 +0000 UTC m=+497.285500624" watchObservedRunningTime="2026-07-22 16:30:54.581008016 +0000 UTC m=+497.286450596" Jul 22 16:31:05.568704 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:05.568666 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xvsmd" Jul 22 16:31:07.423175 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:07.423144 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:07.520337 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:07.520306 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:07.929790 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:07.929758 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:11.233506 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:11.233460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:20.697597 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:20.697553 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:20.830612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:20.830580 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:27.213747 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:27.213703 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:27.310097 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:27.310059 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:27.393598 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:27.393554 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:27.468933 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:27.468843 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:30.343363 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:30.343275 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:30.451284 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:30.451252 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:40.888112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:40.888051 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:31:40.939404 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:31:40.939375 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:03.152379 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:03.152342 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:03.251599 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:03.251567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:03.716368 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:03.716329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:03.815971 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:03.815932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:04.209656 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:04.209624 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:04.305635 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:04.305598 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:04.343039 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:04.343007 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:32:14.072181 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:14.072141 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:14.168560 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:14.168523 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:20.263732 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:20.263685 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:20.318411 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:20.318367 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:25.421177 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:25.421140 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:25.795669 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:25.795637 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:32:40.123288 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:40.123243 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:40.994659 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:40.994625 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:32:45.802230 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:45.802197 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:46.895560 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:46.895531 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:32:51.304902 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:51.304868 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:32:52.893884 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:32:52.893848 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:33:09.130729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:33:09.130645 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:33:11.992809 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:33:11.992774 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:33:18.824599 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:33:18.824562 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:33:21.195251 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:33:21.195213 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:34:22.031500 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:34:22.031465 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:34:22.141063 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:34:22.141026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:34:23.204209 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:34:23.204166 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:34:23.306220 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:34:23.306180 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:34:51.696327 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:34:51.696291 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:35:01.390195 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:01.390156 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:35:11.898347 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:11.898309 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:35:15.733346 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:15.733308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:35:27.909379 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:27.909345 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:35:27.966309 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:27.966267 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:35:30.288969 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:30.288933 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:35:30.389669 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:30.389633 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:35:40.928066 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:35:40.928021 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:36:27.836282 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:36:27.836232 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:36:29.388032 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:36:29.388001 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:36:51.596654 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:36:51.596620 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:37:04.090932 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:04.090895 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:37:08.744537 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:08.744501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:08.844131 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:08.844071 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:08.970230 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:08.970190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:14.862998 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:14.862938 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:14.970029 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:14.969989 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:15.064587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:15.064547 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:15.212510 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:15.212419 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:20.764533 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:20.764495 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:20.868637 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:20.868604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:37:38.289486 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:38.289449 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:37:47.595106 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:47.595051 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:37:58.697505 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:37:58.694798 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:38:06.198215 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:38:06.198181 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:38:17.604709 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:38:17.604675 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:38:28.997601 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:38:28.997561 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:38:40.490632 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:38:40.490596 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:38:50.292986 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:38:50.292949 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:39:03.294368 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:39:03.294280 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:39:15.591046 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:39:15.591011 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:39:27.694308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:39:27.694269 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:39:51.605848 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:39:51.605811 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:40:03.192408 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:40:03.192374 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:40:19.900679 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:40:19.900651 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:40:30.110488 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:40:30.110411 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:40:40.205474 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:40:40.205428 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:40:49.900668 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:40:49.900631 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:40:59.893622 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:40:59.893589 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:41:41.199895 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:41:41.199859 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:42:46.892077 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:42:46.892047 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:42:56.189033 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:42:56.188998 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:43:09.994870 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:43:09.994831 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:43:24.099040 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:43:24.099006 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:43:34.298912 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:43:34.298870 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:43:43.896797 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:43:43.896757 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:43:53.193777 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:43:53.193736 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:44:04.689573 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:44:04.689536 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:44:15.586321 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:44:15.586285 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:44:24.496981 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:44:24.496949 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:44:38.098284 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:44:38.098244 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:44:56.291972 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:44:56.291937 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:45:00.143384 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.143307 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g"] Jul 22 16:45:00.146379 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.146362 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g" Jul 22 16:45:00.149475 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.149455 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sh5d4\"" Jul 22 16:45:00.155313 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.155294 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g" Jul 22 16:45:00.159659 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.159633 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g"] Jul 22 16:45:00.283731 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.283707 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g"] Jul 22 16:45:00.286000 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:45:00.285968 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6912381_5d4e_4370_a408_0a62beab2bd7.slice/crio-3c8ee0654aee1ff13938cc8725b9668d338da502832d706678bc6844843039cb WatchSource:0}: Error finding container 3c8ee0654aee1ff13938cc8725b9668d338da502832d706678bc6844843039cb: Status 404 returned error can't find the container with id 3c8ee0654aee1ff13938cc8725b9668d338da502832d706678bc6844843039cb Jul 22 16:45:00.287878 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.287861 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:45:00.600198 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:00.600162 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g" event={"ID":"a6912381-5d4e-4370-a408-0a62beab2bd7","Type":"ContainerStarted","Data":"3c8ee0654aee1ff13938cc8725b9668d338da502832d706678bc6844843039cb"} Jul 22 16:45:03.615829 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:03.615793 2555 generic.go:356] "Generic (PLEG): container finished" podID="a6912381-5d4e-4370-a408-0a62beab2bd7" containerID="fdc311d17b0998c6c6566fd088f43232cc86c65e53c997e7934257dc5bd350e5" exitCode=0 Jul 22 16:45:03.616212 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:03.615876 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g" event={"ID":"a6912381-5d4e-4370-a408-0a62beab2bd7","Type":"ContainerDied","Data":"fdc311d17b0998c6c6566fd088f43232cc86c65e53c997e7934257dc5bd350e5"} Jul 22 16:45:04.746659 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:04.746635 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g" Jul 22 16:45:05.588330 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:05.588293 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:45:05.624281 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:05.624254 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g" Jul 22 16:45:05.624424 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:05.624252 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745645-fsr9g" event={"ID":"a6912381-5d4e-4370-a408-0a62beab2bd7","Type":"ContainerDied","Data":"3c8ee0654aee1ff13938cc8725b9668d338da502832d706678bc6844843039cb"} Jul 22 16:45:05.624424 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:05.624381 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c8ee0654aee1ff13938cc8725b9668d338da502832d706678bc6844843039cb" Jul 22 16:45:07.083017 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.082985 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp"] Jul 22 16:45:07.083440 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.083282 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6912381-5d4e-4370-a408-0a62beab2bd7" containerName="cleanup" Jul 22 16:45:07.083440 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.083294 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6912381-5d4e-4370-a408-0a62beab2bd7" containerName="cleanup" Jul 22 16:45:07.083440 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.083359 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6912381-5d4e-4370-a408-0a62beab2bd7" containerName="cleanup" Jul 22 16:45:07.086218 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.086200 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.089260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.089216 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 16:45:07.089260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.089220 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wr8qt\"" Jul 22 16:45:07.090603 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.090584 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 16:45:07.090692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.090662 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-1e90b2f6-kserve-self-signed-certs\"" Jul 22 16:45:07.094901 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.094883 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp"] Jul 22 16:45:07.143100 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.143067 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:45:07.225400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.225365 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-model-cache\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.225400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.225398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-tmp-dir\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.225612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.225447 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b1576c-7ec9-44c1-b898-8c3562f892e7-tls-certs\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.225612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.225487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gthlk\" (UniqueName: \"kubernetes.io/projected/07b1576c-7ec9-44c1-b898-8c3562f892e7-kube-api-access-gthlk\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.225612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.225510 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-home\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.225612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.225533 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-dshm\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.326931 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.326891 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-model-cache\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.326931 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.326930 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-tmp-dir\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.327206 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.326983 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b1576c-7ec9-44c1-b898-8c3562f892e7-tls-certs\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.327206 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.327001 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gthlk\" (UniqueName: \"kubernetes.io/projected/07b1576c-7ec9-44c1-b898-8c3562f892e7-kube-api-access-gthlk\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.327206 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.327032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-home\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.327206 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.327062 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-dshm\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.327404 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.327309 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-model-cache\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.327404 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.327361 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-tmp-dir\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.327499 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.327407 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-home\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.329441 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.329420 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-dshm\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.329652 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.329633 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b1576c-7ec9-44c1-b898-8c3562f892e7-tls-certs\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.338648 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.338596 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gthlk\" (UniqueName: \"kubernetes.io/projected/07b1576c-7ec9-44c1-b898-8c3562f892e7-kube-api-access-gthlk\") pod \"e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.395636 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.395605 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:07.523196 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.523170 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp"] Jul 22 16:45:07.525140 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:45:07.525117 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07b1576c_7ec9_44c1_b898_8c3562f892e7.slice/crio-bb7849203d4b6a766cdae7e600e4d4fc4180ca948fa8b5a03429b805474632cb WatchSource:0}: Error finding container bb7849203d4b6a766cdae7e600e4d4fc4180ca948fa8b5a03429b805474632cb: Status 404 returned error can't find the container with id bb7849203d4b6a766cdae7e600e4d4fc4180ca948fa8b5a03429b805474632cb Jul 22 16:45:07.633124 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:07.633025 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" event={"ID":"07b1576c-7ec9-44c1-b898-8c3562f892e7","Type":"ContainerStarted","Data":"bb7849203d4b6a766cdae7e600e4d4fc4180ca948fa8b5a03429b805474632cb"} Jul 22 16:45:09.134339 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:09.134300 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:45:09.643244 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:09.643211 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" event={"ID":"07b1576c-7ec9-44c1-b898-8c3562f892e7","Type":"ContainerStarted","Data":"66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5"} Jul 22 16:45:09.659004 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:09.658877 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" podStartSLOduration=1.568878835 podStartE2EDuration="2.658858447s" totalImagesPullingTime="1.089979612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:45:09.658296106 +0000 UTC m=+1352.363738628" watchObservedRunningTime="2026-07-22 16:45:09.658858447 +0000 UTC m=+1352.364300990" Jul 22 16:45:10.637009 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:10.636980 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:45:10.692513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:10.692481 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:45:17.396520 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:17.396480 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:17.396520 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:17.396532 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:17.409125 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:17.409070 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:17.692376 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:17.692286 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:45:19.803264 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:19.803229 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:45:20.982588 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:20.982550 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq"] Jul 22 16:45:20.986238 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:20.986215 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:20.988946 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:20.988923 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-1e90b2f6-kserve-self-signed-certs\"" Jul 22 16:45:20.997026 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:20.997000 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq"] Jul 22 16:45:21.019939 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.019915 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:45:21.037189 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.037163 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-home\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.037299 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.037202 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tmp-dir\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.037299 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.037224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp4jd\" (UniqueName: \"kubernetes.io/projected/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-kube-api-access-sp4jd\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.037385 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.037300 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-dshm\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.037385 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.037326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-model-cache\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.037385 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.037360 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tls-certs\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.137878 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.137840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-home\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138044 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.137897 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tmp-dir\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138044 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.137928 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp4jd\" (UniqueName: \"kubernetes.io/projected/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-kube-api-access-sp4jd\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138044 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.137967 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-dshm\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138044 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.137999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-model-cache\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138298 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.138046 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tls-certs\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138359 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.138329 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-home\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138414 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.138378 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tmp-dir\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.138467 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.138432 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-model-cache\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.140459 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.140436 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-dshm\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.140685 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.140669 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tls-certs\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.146864 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.146839 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp4jd\" (UniqueName: \"kubernetes.io/projected/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-kube-api-access-sp4jd\") pod \"e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.295708 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.295681 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:21.422646 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.422622 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq"] Jul 22 16:45:21.424887 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:45:21.424863 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d70ecbd_4cc2_4e08_ab07_4821b812b3ae.slice/crio-fa9c2328c568233b6472117b46973a18f883ecb1c964b9c92396de3844586be7 WatchSource:0}: Error finding container fa9c2328c568233b6472117b46973a18f883ecb1c964b9c92396de3844586be7: Status 404 returned error can't find the container with id fa9c2328c568233b6472117b46973a18f883ecb1c964b9c92396de3844586be7 Jul 22 16:45:21.699500 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.699411 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" event={"ID":"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae","Type":"ContainerStarted","Data":"6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317"} Jul 22 16:45:21.699500 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.699454 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" event={"ID":"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae","Type":"ContainerStarted","Data":"fa9c2328c568233b6472117b46973a18f883ecb1c964b9c92396de3844586be7"} Jul 22 16:45:21.715823 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:21.715766 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" podStartSLOduration=1.7157466559999999 podStartE2EDuration="1.715746656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:45:21.715239019 +0000 UTC m=+1364.420681573" watchObservedRunningTime="2026-07-22 16:45:21.715746656 +0000 UTC m=+1364.421189199" Jul 22 16:45:31.296577 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:31.296527 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:31.297052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:31.296601 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:31.309198 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:31.309166 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:31.335477 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:31.335439 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:45:31.751263 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:31.751182 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:45:47.492478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:47.492447 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp"] Jul 22 16:45:47.513708 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:47.513670 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:45:48.290962 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:48.290928 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq"] Jul 22 16:45:48.314912 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:48.314886 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:45:48.995426 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:45:48.995390 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:46:02.512988 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.512933 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" podUID="07b1576c-7ec9-44c1-b898-8c3562f892e7" containerName="main" containerID="cri-o://66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5" gracePeriod=45 Jul 22 16:46:02.761374 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.761349 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:46:02.866883 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.866853 2555 generic.go:356] "Generic (PLEG): container finished" podID="07b1576c-7ec9-44c1-b898-8c3562f892e7" containerID="66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5" exitCode=0 Jul 22 16:46:02.867167 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.866898 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" Jul 22 16:46:02.867167 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.866944 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" event={"ID":"07b1576c-7ec9-44c1-b898-8c3562f892e7","Type":"ContainerDied","Data":"66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5"} Jul 22 16:46:02.867167 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.866975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp" event={"ID":"07b1576c-7ec9-44c1-b898-8c3562f892e7","Type":"ContainerDied","Data":"bb7849203d4b6a766cdae7e600e4d4fc4180ca948fa8b5a03429b805474632cb"} Jul 22 16:46:02.867167 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.866991 2555 scope.go:117] "RemoveContainer" containerID="66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5" Jul 22 16:46:02.867482 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867466 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-model-cache\") pod \"07b1576c-7ec9-44c1-b898-8c3562f892e7\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " Jul 22 16:46:02.867534 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867496 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-dshm\") pod \"07b1576c-7ec9-44c1-b898-8c3562f892e7\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " Jul 22 16:46:02.867649 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867579 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b1576c-7ec9-44c1-b898-8c3562f892e7-tls-certs\") pod \"07b1576c-7ec9-44c1-b898-8c3562f892e7\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " Jul 22 16:46:02.867649 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867634 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gthlk\" (UniqueName: \"kubernetes.io/projected/07b1576c-7ec9-44c1-b898-8c3562f892e7-kube-api-access-gthlk\") pod \"07b1576c-7ec9-44c1-b898-8c3562f892e7\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " Jul 22 16:46:02.867757 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867674 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-tmp-dir\") pod \"07b1576c-7ec9-44c1-b898-8c3562f892e7\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " Jul 22 16:46:02.867952 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867808 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-home\") pod \"07b1576c-7ec9-44c1-b898-8c3562f892e7\" (UID: \"07b1576c-7ec9-44c1-b898-8c3562f892e7\") " Jul 22 16:46:02.867952 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867828 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "07b1576c-7ec9-44c1-b898-8c3562f892e7" (UID: "07b1576c-7ec9-44c1-b898-8c3562f892e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:02.867952 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.867915 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07b1576c-7ec9-44c1-b898-8c3562f892e7" (UID: "07b1576c-7ec9-44c1-b898-8c3562f892e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:02.868395 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.868079 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-home" (OuterVolumeSpecName: "home") pod "07b1576c-7ec9-44c1-b898-8c3562f892e7" (UID: "07b1576c-7ec9-44c1-b898-8c3562f892e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:02.868395 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.868222 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-tmp-dir\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:02.868395 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.868241 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-home\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:02.868395 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.868254 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-model-cache\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:02.869939 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.869914 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-dshm" (OuterVolumeSpecName: "dshm") pod "07b1576c-7ec9-44c1-b898-8c3562f892e7" (UID: "07b1576c-7ec9-44c1-b898-8c3562f892e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:02.870039 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.870017 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b1576c-7ec9-44c1-b898-8c3562f892e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07b1576c-7ec9-44c1-b898-8c3562f892e7" (UID: "07b1576c-7ec9-44c1-b898-8c3562f892e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:46:02.870107 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.870033 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b1576c-7ec9-44c1-b898-8c3562f892e7-kube-api-access-gthlk" (OuterVolumeSpecName: "kube-api-access-gthlk") pod "07b1576c-7ec9-44c1-b898-8c3562f892e7" (UID: "07b1576c-7ec9-44c1-b898-8c3562f892e7"). InnerVolumeSpecName "kube-api-access-gthlk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:46:02.883735 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.883716 2555 scope.go:117] "RemoveContainer" containerID="66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5" Jul 22 16:46:02.884024 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:46:02.884007 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5\": container with ID starting with 66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5 not found: ID does not exist" containerID="66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5" Jul 22 16:46:02.884064 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.884032 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5"} err="failed to get container status \"66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5\": rpc error: code = NotFound desc = could not find container \"66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5\": container with ID starting with 66e41f7a908510098fd31ea03cbb13847ef84cbd37bcadb83d36d4dcf8c175f5 not found: ID does not exist" Jul 22 16:46:02.969521 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.969492 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07b1576c-7ec9-44c1-b898-8c3562f892e7-dshm\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:02.969521 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.969514 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07b1576c-7ec9-44c1-b898-8c3562f892e7-tls-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:02.969521 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:02.969524 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gthlk\" (UniqueName: \"kubernetes.io/projected/07b1576c-7ec9-44c1-b898-8c3562f892e7-kube-api-access-gthlk\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:03.187951 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.187921 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp"] Jul 22 16:46:03.191940 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.191918 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-1e90b2f6-kserve-6777446955-z2mjp"] Jul 22 16:46:03.311395 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.311353 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" podUID="9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" containerName="main" containerID="cri-o://6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317" gracePeriod=45 Jul 22 16:46:03.561143 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.561122 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:46:03.674833 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.674805 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-dshm\") pod \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " Jul 22 16:46:03.674978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.674854 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-home\") pod \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " Jul 22 16:46:03.674978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.674876 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tls-certs\") pod \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " Jul 22 16:46:03.674978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.674917 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tmp-dir\") pod \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " Jul 22 16:46:03.674978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.674967 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp4jd\" (UniqueName: \"kubernetes.io/projected/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-kube-api-access-sp4jd\") pod \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " Jul 22 16:46:03.675181 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.674992 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-model-cache\") pod \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\" (UID: \"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae\") " Jul 22 16:46:03.675181 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.675126 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-home" (OuterVolumeSpecName: "home") pod "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" (UID: "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:03.675283 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.675223 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-home\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:03.675283 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.675216 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" (UID: "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:03.675375 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.675308 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" (UID: "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:03.676975 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.676948 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-dshm" (OuterVolumeSpecName: "dshm") pod "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" (UID: "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:03.677166 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.677143 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" (UID: "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:46:03.677238 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.677219 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-kube-api-access-sp4jd" (OuterVolumeSpecName: "kube-api-access-sp4jd") pod "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" (UID: "9d70ecbd-4cc2-4e08-ab07-4821b812b3ae"). InnerVolumeSpecName "kube-api-access-sp4jd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:46:03.743634 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.743603 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b1576c-7ec9-44c1-b898-8c3562f892e7" path="/var/lib/kubelet/pods/07b1576c-7ec9-44c1-b898-8c3562f892e7/volumes" Jul 22 16:46:03.776397 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.776371 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-dshm\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:03.776397 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.776398 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tls-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:03.776528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.776411 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-tmp-dir\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:03.776528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.776420 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp4jd\" (UniqueName: \"kubernetes.io/projected/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-kube-api-access-sp4jd\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:03.776528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.776428 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae-model-cache\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:46:03.872932 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.872900 2555 generic.go:356] "Generic (PLEG): container finished" podID="9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" containerID="6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317" exitCode=0 Jul 22 16:46:03.873055 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.872950 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" Jul 22 16:46:03.873055 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.872960 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" event={"ID":"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae","Type":"ContainerDied","Data":"6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317"} Jul 22 16:46:03.873055 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.873000 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq" event={"ID":"9d70ecbd-4cc2-4e08-ab07-4821b812b3ae","Type":"ContainerDied","Data":"fa9c2328c568233b6472117b46973a18f883ecb1c964b9c92396de3844586be7"} Jul 22 16:46:03.873055 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.873023 2555 scope.go:117] "RemoveContainer" containerID="6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317" Jul 22 16:46:03.880975 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.880958 2555 scope.go:117] "RemoveContainer" containerID="6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317" Jul 22 16:46:03.881247 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:46:03.881225 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317\": container with ID starting with 6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317 not found: ID does not exist" containerID="6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317" Jul 22 16:46:03.881322 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.881254 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317"} err="failed to get container status \"6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317\": rpc error: code = NotFound desc = could not find container \"6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317\": container with ID starting with 6b642062a1e77e91a16cdf41d0a55ae8aa7951d638e6a1e1b36f5ff0bd386317 not found: ID does not exist" Jul 22 16:46:03.887854 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.887832 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq"] Jul 22 16:46:03.891789 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:03.891770 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-1e90b2f6-kserve-56f4555699-xr5pq"] Jul 22 16:46:04.084515 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:04.084480 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:46:05.748626 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:05.748583 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" path="/var/lib/kubelet/pods/9d70ecbd-4cc2-4e08-ab07-4821b812b3ae/volumes" Jul 22 16:46:11.902863 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:11.902827 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:46:24.401424 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:24.401386 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:46:35.497743 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:35.497711 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:46:45.099081 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:45.099043 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:46:52.798487 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:46:52.798446 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:47:04.294643 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:04.294605 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:47:08.010791 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:08.010754 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:47:14.094043 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:14.094007 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:47:19.549329 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:19.549281 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:47:24.587955 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:24.587919 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:47:36.897832 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:36.897803 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:47:52.189733 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:52.189693 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:47:59.894507 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:47:59.894426 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:48:15.498052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:48:15.498008 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:48:27.300740 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:48:27.300703 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:48:39.798601 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:48:39.798562 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:48:53.092859 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:48:53.092826 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:48:56.694298 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:48:56.694257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:03.404691 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:03.404653 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:06.794687 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:06.794650 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:49:10.100379 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:10.100321 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:10.200492 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:10.200457 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:11.685082 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:11.685045 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:15.794478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:15.794445 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:49:16.997108 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:16.997062 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:20.153383 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:20.153346 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:20.252510 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:20.252475 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:20.624798 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:20.624757 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:25.796316 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:25.796288 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:49:26.265604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.265512 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:26.325134 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.325070 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:26.490535 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.490504 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:26.771944 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.771903 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl"] Jul 22 16:49:26.772377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.772357 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b1576c-7ec9-44c1-b898-8c3562f892e7" containerName="main" Jul 22 16:49:26.772551 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.772530 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b1576c-7ec9-44c1-b898-8c3562f892e7" containerName="main" Jul 22 16:49:26.772641 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.772557 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" containerName="main" Jul 22 16:49:26.772641 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.772566 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" containerName="main" Jul 22 16:49:26.772712 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.772679 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b1576c-7ec9-44c1-b898-8c3562f892e7" containerName="main" Jul 22 16:49:26.772712 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.772690 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d70ecbd-4cc2-4e08-ab07-4821b812b3ae" containerName="main" Jul 22 16:49:26.775824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.775806 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.778636 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.778615 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-60e975a1-openshift-default-dockercfg-kn4b6\"" Jul 22 16:49:26.794817 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.794790 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl"] Jul 22 16:49:26.804572 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.804545 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:26.870622 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870586 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-credential-socket\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870782 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870632 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-data\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870782 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870664 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-token\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870782 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870692 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-envoy\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870782 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870709 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-podinfo\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870782 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58nk4\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-kube-api-access-58nk4\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870969 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870791 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-socket\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870969 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870814 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caac5c62-699d-425c-906f-ac9348a4ad6f-istiod-ca-cert\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.870969 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.870853 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-certs\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.874983 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.874957 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:26.972164 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972129 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-socket\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972164 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caac5c62-699d-425c-906f-ac9348a4ad6f-istiod-ca-cert\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972421 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972205 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-certs\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972421 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-credential-socket\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972421 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972274 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-data\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972576 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972418 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-token\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972576 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972479 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-envoy\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972576 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972511 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-podinfo\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972576 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972532 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58nk4\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-kube-api-access-58nk4\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972764 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972600 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-socket\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972764 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972627 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-certs\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972764 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972706 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-credential-socket\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.972764 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.972727 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-data\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.973034 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.973013 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caac5c62-699d-425c-906f-ac9348a4ad6f-istiod-ca-cert\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.974927 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.974903 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-envoy\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.975044 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.975013 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-podinfo\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.980902 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.980883 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-token\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:26.981211 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:26.981190 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58nk4\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-kube-api-access-58nk4\") pod \"e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:27.087536 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.087504 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:27.232136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.232108 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl"] Jul 22 16:49:27.234241 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:49:27.234215 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaac5c62_699d_425c_906f_ac9348a4ad6f.slice/crio-b843a70cc7b0e8591f6770fe9da860bf6b2f3c5b0db4871cd41f8708bdbd52ef WatchSource:0}: Error finding container b843a70cc7b0e8591f6770fe9da860bf6b2f3c5b0db4871cd41f8708bdbd52ef: Status 404 returned error can't find the container with id b843a70cc7b0e8591f6770fe9da860bf6b2f3c5b0db4871cd41f8708bdbd52ef Jul 22 16:49:27.236303 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.236272 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:49:27.236394 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.236342 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:49:27.236394 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.236376 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:49:27.675853 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.675814 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" event={"ID":"caac5c62-699d-425c-906f-ac9348a4ad6f","Type":"ContainerStarted","Data":"70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9"} Jul 22 16:49:27.676036 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.675882 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" event={"ID":"caac5c62-699d-425c-906f-ac9348a4ad6f","Type":"ContainerStarted","Data":"b843a70cc7b0e8591f6770fe9da860bf6b2f3c5b0db4871cd41f8708bdbd52ef"} Jul 22 16:49:27.699621 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:27.699576 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" podStartSLOduration=1.6995603799999999 podStartE2EDuration="1.69956038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:49:27.697204256 +0000 UTC m=+1610.402646798" watchObservedRunningTime="2026-07-22 16:49:27.69956038 +0000 UTC m=+1610.405002990" Jul 22 16:49:28.088111 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:28.088059 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:28.092807 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:28.092784 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:28.679633 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:28.679601 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:28.680499 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:28.680480 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:29.113375 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:29.113341 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:29.211958 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:29.211927 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:29.315331 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:29.315298 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:29.708706 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:29.708669 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:29.904728 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:29.904686 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:30.006695 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:30.006597 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:49:36.997831 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:36.997795 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:49:44.644773 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.643731 2555 patch_prober.go:28] interesting pod/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": read tcp 10.133.0.2:59008->10.133.0.23:15021: read: connection reset by peer" start-of-body= Jul 22 16:49:44.644773 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.643819 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": read tcp 10.133.0.2:59008->10.133.0.23:15021: read: connection reset by peer" Jul 22 16:49:44.754425 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.754395 2555 generic.go:356] "Generic (PLEG): container finished" podID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerID="70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9" exitCode=0 Jul 22 16:49:44.754579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.754465 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" event={"ID":"caac5c62-699d-425c-906f-ac9348a4ad6f","Type":"ContainerDied","Data":"70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9"} Jul 22 16:49:44.754778 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.754764 2555 scope.go:117] "RemoveContainer" containerID="70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9" Jul 22 16:49:44.755683 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.755660 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:49:44.755736 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.755727 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:49:44.755769 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:44.755759 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:49:45.760433 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:45.760392 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" event={"ID":"caac5c62-699d-425c-906f-ac9348a4ad6f","Type":"ContainerStarted","Data":"86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4"} Jul 22 16:49:46.088538 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:46.088502 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:46.088744 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:46.088722 2555 patch_prober.go:28] interesting pod/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jul 22 16:49:46.088796 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:46.088772 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jul 22 16:49:47.088022 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:47.087985 2555 patch_prober.go:28] interesting pod/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jul 22 16:49:47.088523 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:47.088071 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jul 22 16:49:48.088917 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:48.088877 2555 patch_prober.go:28] interesting pod/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jul 22 16:49:48.089337 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:48.088935 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jul 22 16:49:49.088101 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:49.088062 2555 patch_prober.go:28] interesting pod/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jul 22 16:49:49.088254 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:49.088150 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jul 22 16:49:50.091692 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:50.091658 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:50.092066 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:50.092007 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:50.092763 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:50.092748 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:49:54.621635 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.621600 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt"] Jul 22 16:49:54.625368 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.625353 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" Jul 22 16:49:54.627810 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.627791 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 16:49:54.627932 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.627867 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sh5d4\"" Jul 22 16:49:54.631202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.631180 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt"] Jul 22 16:49:54.709074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.709046 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504f5724-13b0-41ae-8e1a-db474b863f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60e975a1-tbszt\" (UID: \"504f5724-13b0-41ae-8e1a-db474b863f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" Jul 22 16:49:54.809495 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.809463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504f5724-13b0-41ae-8e1a-db474b863f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60e975a1-tbszt\" (UID: \"504f5724-13b0-41ae-8e1a-db474b863f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" Jul 22 16:49:54.810036 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.810016 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504f5724-13b0-41ae-8e1a-db474b863f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60e975a1-tbszt\" (UID: \"504f5724-13b0-41ae-8e1a-db474b863f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" Jul 22 16:49:54.937305 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:54.937213 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" Jul 22 16:49:55.064455 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:55.064394 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt"] Jul 22 16:49:55.066604 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:49:55.066575 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504f5724_13b0_41ae_8e1a_db474b863f9f.slice/crio-0d64a3fbe73d6fc4da4d285539f52082490a04fabca6945ea0a75737829b7f9d WatchSource:0}: Error finding container 0d64a3fbe73d6fc4da4d285539f52082490a04fabca6945ea0a75737829b7f9d: Status 404 returned error can't find the container with id 0d64a3fbe73d6fc4da4d285539f52082490a04fabca6945ea0a75737829b7f9d Jul 22 16:49:55.805016 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:55.804980 2555 generic.go:356] "Generic (PLEG): container finished" podID="504f5724-13b0-41ae-8e1a-db474b863f9f" containerID="ded0259e579a4e06a824190be28a6330776f92410dbffc1f000d331fd9a712e3" exitCode=0 Jul 22 16:49:55.805443 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:55.805073 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" event={"ID":"504f5724-13b0-41ae-8e1a-db474b863f9f","Type":"ContainerDied","Data":"ded0259e579a4e06a824190be28a6330776f92410dbffc1f000d331fd9a712e3"} Jul 22 16:49:55.805443 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:55.805160 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" event={"ID":"504f5724-13b0-41ae-8e1a-db474b863f9f","Type":"ContainerStarted","Data":"0d64a3fbe73d6fc4da4d285539f52082490a04fabca6945ea0a75737829b7f9d"} Jul 22 16:49:56.938407 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:56.938385 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" Jul 22 16:49:57.026001 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:57.025970 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504f5724-13b0-41ae-8e1a-db474b863f9f-maas-api-service-ca\") pod \"504f5724-13b0-41ae-8e1a-db474b863f9f\" (UID: \"504f5724-13b0-41ae-8e1a-db474b863f9f\") " Jul 22 16:49:57.026358 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:57.026336 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504f5724-13b0-41ae-8e1a-db474b863f9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "504f5724-13b0-41ae-8e1a-db474b863f9f" (UID: "504f5724-13b0-41ae-8e1a-db474b863f9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:49:57.126969 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:57.126883 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504f5724-13b0-41ae-8e1a-db474b863f9f-maas-api-service-ca\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:49:57.814443 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:57.814407 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" event={"ID":"504f5724-13b0-41ae-8e1a-db474b863f9f","Type":"ContainerDied","Data":"0d64a3fbe73d6fc4da4d285539f52082490a04fabca6945ea0a75737829b7f9d"} Jul 22 16:49:57.814612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:57.814460 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d64a3fbe73d6fc4da4d285539f52082490a04fabca6945ea0a75737829b7f9d" Jul 22 16:49:57.814612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:49:57.814457 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt" Jul 22 16:50:05.138287 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:05.138238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:08.393178 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:08.393148 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:50:09.690679 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:09.690638 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:09.693992 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:09.693970 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt"] Jul 22 16:50:09.696453 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:09.696433 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60e975a1-tbszt"] Jul 22 16:50:09.743754 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:09.743725 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504f5724-13b0-41ae-8e1a-db474b863f9f" path="/var/lib/kubelet/pods/504f5724-13b0-41ae-8e1a-db474b863f9f/volumes" Jul 22 16:50:09.834467 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:09.834438 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl"] Jul 22 16:50:09.834673 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:09.834651 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" containerID="cri-o://86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4" gracePeriod=30 Jul 22 16:50:11.004997 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.004962 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7"] Jul 22 16:50:11.005377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.005250 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504f5724-13b0-41ae-8e1a-db474b863f9f" containerName="revoke-keys" Jul 22 16:50:11.005377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.005261 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="504f5724-13b0-41ae-8e1a-db474b863f9f" containerName="revoke-keys" Jul 22 16:50:11.005377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.005313 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="504f5724-13b0-41ae-8e1a-db474b863f9f" containerName="revoke-keys" Jul 22 16:50:11.008244 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.008227 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.012617 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.012599 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c45d36b4-gw-openshift-default-dockercfg-9mspj\"" Jul 22 16:50:11.027622 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.027597 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7"] Jul 22 16:50:11.141599 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141570 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-envoy\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141766 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141611 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz8ns\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-kube-api-access-mz8ns\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141766 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141646 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-credential-socket\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141766 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-istio-token\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141766 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141754 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-certs\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141937 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141778 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-data\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141937 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141836 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ced37791-c686-4b65-91d7-631a375ca538-istiod-ca-cert\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141937 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141866 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-socket\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.141937 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.141880 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ced37791-c686-4b65-91d7-631a375ca538-istio-podinfo\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.230027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.229981 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:11.242993 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.242950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-data\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243155 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243026 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ced37791-c686-4b65-91d7-631a375ca538-istiod-ca-cert\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243155 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243065 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-socket\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243155 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243116 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ced37791-c686-4b65-91d7-631a375ca538-istio-podinfo\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243155 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-envoy\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243367 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz8ns\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-kube-api-access-mz8ns\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243367 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243205 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-credential-socket\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243367 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243232 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-istio-token\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243367 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243265 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-certs\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243368 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-data\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243391 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-socket\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243548 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-certs\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243696 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243608 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-credential-socket\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.243818 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.243793 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ced37791-c686-4b65-91d7-631a375ca538-istiod-ca-cert\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.245602 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.245582 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-envoy\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.245814 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.245795 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ced37791-c686-4b65-91d7-631a375ca538-istio-podinfo\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.251547 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.251516 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-istio-token\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.252078 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.252054 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz8ns\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-kube-api-access-mz8ns\") pod \"e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.317971 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.317940 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:11.453026 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.453002 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7"] Jul 22 16:50:11.455630 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:50:11.455603 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced37791_c686_4b65_91d7_631a375ca538.slice/crio-9f36c5d720e38e19c447d29a2f358aa8329e37ba5add64cfb3e603d40ef8809e WatchSource:0}: Error finding container 9f36c5d720e38e19c447d29a2f358aa8329e37ba5add64cfb3e603d40ef8809e: Status 404 returned error can't find the container with id 9f36c5d720e38e19c447d29a2f358aa8329e37ba5add64cfb3e603d40ef8809e Jul 22 16:50:11.457650 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.457620 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:11.457731 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.457684 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:11.457731 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.457708 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:11.872015 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.871957 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" event={"ID":"ced37791-c686-4b65-91d7-631a375ca538","Type":"ContainerStarted","Data":"21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2"} Jul 22 16:50:11.872218 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.872042 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" event={"ID":"ced37791-c686-4b65-91d7-631a375ca538","Type":"ContainerStarted","Data":"9f36c5d720e38e19c447d29a2f358aa8329e37ba5add64cfb3e603d40ef8809e"} Jul 22 16:50:11.893892 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:11.893838 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" podStartSLOduration=1.893823547 podStartE2EDuration="1.893823547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:50:11.893302807 +0000 UTC m=+1654.598745349" watchObservedRunningTime="2026-07-22 16:50:11.893823547 +0000 UTC m=+1654.599266088" Jul 22 16:50:12.318199 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:12.318156 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:12.322714 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:12.322690 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:12.876450 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:12.876420 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:12.877415 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:12.877393 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:13.344384 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:13.344350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:13.452807 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:13.452759 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:13.575245 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:13.575210 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:13.947621 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:13.947562 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:14.051746 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:14.051707 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:15.079111 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.079071 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:50:15.179356 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179269 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-data\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179356 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179307 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-socket\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179356 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179343 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-token\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179399 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58nk4\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-kube-api-access-58nk4\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179450 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-podinfo\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179483 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-credential-socket\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179510 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-envoy\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179546 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-certs\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179572 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caac5c62-699d-425c-906f-ac9348a4ad6f-istiod-ca-cert\") pod \"caac5c62-699d-425c-906f-ac9348a4ad6f\" (UID: \"caac5c62-699d-425c-906f-ac9348a4ad6f\") " Jul 22 16:50:15.179703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179621 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:15.180046 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179808 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.180046 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.179882 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:15.180046 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.180000 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:15.180046 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.180019 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caac5c62-699d-425c-906f-ac9348a4ad6f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:50:15.180446 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.180239 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-data" (OuterVolumeSpecName: "istio-data") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:15.181992 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.181968 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-kube-api-access-58nk4" (OuterVolumeSpecName: "kube-api-access-58nk4") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "kube-api-access-58nk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:15.181992 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.181977 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 16:50:15.182214 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.182199 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:15.182312 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.182300 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-token" (OuterVolumeSpecName: "istio-token") pod "caac5c62-699d-425c-906f-ac9348a4ad6f" (UID: "caac5c62-699d-425c-906f-ac9348a4ad6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:15.280676 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280646 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.280676 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280672 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caac5c62-699d-425c-906f-ac9348a4ad6f-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.280676 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280681 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.280916 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280690 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.280916 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280700 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58nk4\" (UniqueName: \"kubernetes.io/projected/caac5c62-699d-425c-906f-ac9348a4ad6f-kube-api-access-58nk4\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.280916 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280708 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.280916 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280716 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.280916 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.280724 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caac5c62-699d-425c-906f-ac9348a4ad6f-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:15.889642 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.889606 2555 generic.go:356] "Generic (PLEG): container finished" podID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerID="86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4" exitCode=0 Jul 22 16:50:15.889808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.889653 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" Jul 22 16:50:15.889808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.889682 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" event={"ID":"caac5c62-699d-425c-906f-ac9348a4ad6f","Type":"ContainerDied","Data":"86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4"} Jul 22 16:50:15.889808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.889723 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl" event={"ID":"caac5c62-699d-425c-906f-ac9348a4ad6f","Type":"ContainerDied","Data":"b843a70cc7b0e8591f6770fe9da860bf6b2f3c5b0db4871cd41f8708bdbd52ef"} Jul 22 16:50:15.889808 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.889744 2555 scope.go:117] "RemoveContainer" containerID="86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4" Jul 22 16:50:15.898253 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.898229 2555 scope.go:117] "RemoveContainer" containerID="70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9" Jul 22 16:50:15.905423 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.905393 2555 scope.go:117] "RemoveContainer" containerID="86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4" Jul 22 16:50:15.905656 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:50:15.905638 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4\": container with ID starting with 86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4 not found: ID does not exist" containerID="86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4" Jul 22 16:50:15.905738 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.905669 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4"} err="failed to get container status \"86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4\": rpc error: code = NotFound desc = could not find container \"86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4\": container with ID starting with 86ed558a5ac00298083dab099aab83cd921369d19ba877750509e0d1cff345d4 not found: ID does not exist" Jul 22 16:50:15.905738 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.905686 2555 scope.go:117] "RemoveContainer" containerID="70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9" Jul 22 16:50:15.905928 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:50:15.905911 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9\": container with ID starting with 70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9 not found: ID does not exist" containerID="70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9" Jul 22 16:50:15.905999 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.905932 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9"} err="failed to get container status \"70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9\": rpc error: code = NotFound desc = could not find container \"70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9\": container with ID starting with 70e6b9be8268162e9946f93bc6d363d1b3bf37616a5cddac967a477aaaa4b3b9 not found: ID does not exist" Jul 22 16:50:15.908234 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.908213 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl"] Jul 22 16:50:15.910948 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:15.910926 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-60e975a1-openshift-default-b474456dc-tsxdl"] Jul 22 16:50:17.744052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:17.744020 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" path="/var/lib/kubelet/pods/caac5c62-699d-425c-906f-ac9348a4ad6f/volumes" Jul 22 16:50:19.598747 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:19.598713 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:50:27.878524 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.878487 2555 patch_prober.go:28] interesting pod/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 22 16:50:27.878961 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.878555 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 22 16:50:27.956156 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.956125 2555 generic.go:356] "Generic (PLEG): container finished" podID="ced37791-c686-4b65-91d7-631a375ca538" containerID="21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2" exitCode=0 Jul 22 16:50:27.956323 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.956201 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" event={"ID":"ced37791-c686-4b65-91d7-631a375ca538","Type":"ContainerDied","Data":"21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2"} Jul 22 16:50:27.956588 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.956573 2555 scope.go:117] "RemoveContainer" containerID="21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2" Jul 22 16:50:27.957550 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.957524 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:27.957631 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.957585 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:27.957631 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:27.957618 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:28.961926 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:28.961887 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" event={"ID":"ced37791-c686-4b65-91d7-631a375ca538","Type":"ContainerStarted","Data":"31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd"} Jul 22 16:50:29.318431 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:29.318387 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:29.318648 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:29.318525 2555 patch_prober.go:28] interesting pod/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 22 16:50:29.318648 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:29.318572 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 22 16:50:30.318984 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:30.318944 2555 patch_prober.go:28] interesting pod/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 22 16:50:30.319623 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:30.319590 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 22 16:50:31.319494 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:31.319444 2555 patch_prober.go:28] interesting pod/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 22 16:50:31.319999 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:31.319522 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 22 16:50:32.319506 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:32.318934 2555 patch_prober.go:28] interesting pod/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 22 16:50:32.319506 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:32.319006 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 22 16:50:33.322320 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:33.322286 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:33.322791 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:33.322725 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:33.323365 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:33.323347 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:38.367117 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.367056 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq"] Jul 22 16:50:38.367513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.367402 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" Jul 22 16:50:38.367513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.367414 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" Jul 22 16:50:38.367513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.367431 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" Jul 22 16:50:38.367513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.367436 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" Jul 22 16:50:38.367513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.367478 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" Jul 22 16:50:38.367513 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.367487 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="caac5c62-699d-425c-906f-ac9348a4ad6f" containerName="istio-proxy" Jul 22 16:50:38.370287 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.370270 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" Jul 22 16:50:38.373053 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.373027 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 16:50:38.373150 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.373134 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sh5d4\"" Jul 22 16:50:38.378334 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.378310 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq"] Jul 22 16:50:38.485539 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.485487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de25025a-28f6-448b-b857-adb8bb14d86c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq\" (UID: \"de25025a-28f6-448b-b857-adb8bb14d86c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" Jul 22 16:50:38.586337 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.586300 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de25025a-28f6-448b-b857-adb8bb14d86c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq\" (UID: \"de25025a-28f6-448b-b857-adb8bb14d86c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" Jul 22 16:50:38.586909 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.586888 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de25025a-28f6-448b-b857-adb8bb14d86c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq\" (UID: \"de25025a-28f6-448b-b857-adb8bb14d86c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" Jul 22 16:50:38.681700 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.681600 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" Jul 22 16:50:38.813465 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:38.813317 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq"] Jul 22 16:50:38.815768 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:50:38.815743 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde25025a_28f6_448b_b857_adb8bb14d86c.slice/crio-40db5c52cd49f161f2a901bd2ef0835b11540a8044baf3a703e01a25160ac146 WatchSource:0}: Error finding container 40db5c52cd49f161f2a901bd2ef0835b11540a8044baf3a703e01a25160ac146: Status 404 returned error can't find the container with id 40db5c52cd49f161f2a901bd2ef0835b11540a8044baf3a703e01a25160ac146 Jul 22 16:50:39.005609 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:39.005522 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" event={"ID":"de25025a-28f6-448b-b857-adb8bb14d86c","Type":"ContainerStarted","Data":"40db5c52cd49f161f2a901bd2ef0835b11540a8044baf3a703e01a25160ac146"} Jul 22 16:50:40.011209 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:40.011168 2555 generic.go:356] "Generic (PLEG): container finished" podID="de25025a-28f6-448b-b857-adb8bb14d86c" containerID="d65daec75ceafa5c265fb8c9269844632102e6614d1c4e59ff8fa7b0bf743ff2" exitCode=0 Jul 22 16:50:40.011684 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:40.011239 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" event={"ID":"de25025a-28f6-448b-b857-adb8bb14d86c","Type":"ContainerDied","Data":"d65daec75ceafa5c265fb8c9269844632102e6614d1c4e59ff8fa7b0bf743ff2"} Jul 22 16:50:41.160339 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:41.160307 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" Jul 22 16:50:41.224873 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:41.224842 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de25025a-28f6-448b-b857-adb8bb14d86c-maas-api-service-ca\") pod \"de25025a-28f6-448b-b857-adb8bb14d86c\" (UID: \"de25025a-28f6-448b-b857-adb8bb14d86c\") " Jul 22 16:50:41.225216 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:41.225197 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de25025a-28f6-448b-b857-adb8bb14d86c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de25025a-28f6-448b-b857-adb8bb14d86c" (UID: "de25025a-28f6-448b-b857-adb8bb14d86c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:50:41.325763 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:41.325724 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de25025a-28f6-448b-b857-adb8bb14d86c-maas-api-service-ca\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:42.032182 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:42.032156 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" Jul 22 16:50:42.032182 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:42.032166 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq" event={"ID":"de25025a-28f6-448b-b857-adb8bb14d86c","Type":"ContainerDied","Data":"40db5c52cd49f161f2a901bd2ef0835b11540a8044baf3a703e01a25160ac146"} Jul 22 16:50:42.032383 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:42.032216 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40db5c52cd49f161f2a901bd2ef0835b11540a8044baf3a703e01a25160ac146" Jul 22 16:50:48.779564 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:48.779527 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:51.995812 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:51.995776 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:50:53.439021 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.438987 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq"] Jul 22 16:50:53.440647 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.440620 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c45d36b4-7xjzq"] Jul 22 16:50:53.454346 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.454317 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:53.611332 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.611293 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7"] Jul 22 16:50:53.611553 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.611528 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" containerID="cri-o://31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd" gracePeriod=30 Jul 22 16:50:53.633043 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.633006 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:53.744100 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.744009 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de25025a-28f6-448b-b857-adb8bb14d86c" path="/var/lib/kubelet/pods/de25025a-28f6-448b-b857-adb8bb14d86c/volumes" Jul 22 16:50:53.917931 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:53.917892 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:54.156988 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.156948 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:54.511604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.511524 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7"] Jul 22 16:50:54.511938 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.511826 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de25025a-28f6-448b-b857-adb8bb14d86c" containerName="revoke-keys" Jul 22 16:50:54.511938 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.511836 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="de25025a-28f6-448b-b857-adb8bb14d86c" containerName="revoke-keys" Jul 22 16:50:54.511938 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.511884 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="de25025a-28f6-448b-b857-adb8bb14d86c" containerName="revoke-keys" Jul 22 16:50:54.515850 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.515829 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.518451 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.518434 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3000a004-openshift-default-dockercfg-l5d2x\"" Jul 22 16:50:54.525161 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.525138 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7"] Jul 22 16:50:54.535422 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.535395 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:54.631932 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.631897 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-envoy\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.631944 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-credential-socket\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.631973 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-istio-token\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.632028 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-certs\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.632052 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7805e5de-e203-4110-b7a0-09117de54fe8-istio-podinfo\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632136 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.632123 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-socket\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632313 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.632174 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-data\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632313 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.632202 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7805e5de-e203-4110-b7a0-09117de54fe8-istiod-ca-cert\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.632313 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.632228 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5pdf\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-kube-api-access-f5pdf\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733344 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733310 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5pdf\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-kube-api-access-f5pdf\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733485 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733365 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-envoy\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733485 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733399 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-credential-socket\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733485 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733414 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-istio-token\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733485 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733443 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-certs\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733485 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733469 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7805e5de-e203-4110-b7a0-09117de54fe8-istio-podinfo\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733750 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733649 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-socket\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733750 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733710 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-data\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733858 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733753 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7805e5de-e203-4110-b7a0-09117de54fe8-istiod-ca-cert\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733914 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733865 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-credential-socket\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.733982 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.733944 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-certs\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.734052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.734019 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-socket\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.734206 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.734054 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-data\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.734467 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.734445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7805e5de-e203-4110-b7a0-09117de54fe8-istiod-ca-cert\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.735894 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.735864 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-envoy\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.736147 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.736131 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7805e5de-e203-4110-b7a0-09117de54fe8-istio-podinfo\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.741498 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.741474 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-istio-token\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.741612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.741511 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5pdf\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-kube-api-access-f5pdf\") pod \"e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.826682 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.826657 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:54.961010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.960977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:54.969360 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.969264 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7"] Jul 22 16:50:54.972024 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:50:54.971986 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7805e5de_e203_4110_b7a0_09117de54fe8.slice/crio-f0012c4d9e8628c351f6dd5f461a325ab1d780f467652696395768d04618e75d WatchSource:0}: Error finding container f0012c4d9e8628c351f6dd5f461a325ab1d780f467652696395768d04618e75d: Status 404 returned error can't find the container with id f0012c4d9e8628c351f6dd5f461a325ab1d780f467652696395768d04618e75d Jul 22 16:50:54.974113 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.974055 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:54.974188 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.974148 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:54.974226 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:54.974196 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:50:55.088381 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:55.088289 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" event={"ID":"7805e5de-e203-4110-b7a0-09117de54fe8","Type":"ContainerStarted","Data":"b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7"} Jul 22 16:50:55.088381 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:55.088351 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" event={"ID":"7805e5de-e203-4110-b7a0-09117de54fe8","Type":"ContainerStarted","Data":"f0012c4d9e8628c351f6dd5f461a325ab1d780f467652696395768d04618e75d"} Jul 22 16:50:55.112256 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:55.112207 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" podStartSLOduration=1.112189163 podStartE2EDuration="1.112189163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:50:55.110937175 +0000 UTC m=+1697.816379716" watchObservedRunningTime="2026-07-22 16:50:55.112189163 +0000 UTC m=+1697.817631684" Jul 22 16:50:55.826755 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:55.826711 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:56.471655 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:56.471615 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:56.543468 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:56.543430 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:56.615716 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:56.615684 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:56.745688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:56.745609 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:56.832499 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:56.832473 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:57.069041 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:57.069013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:57.096010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:57.095984 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:57.096928 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:57.096913 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:50:57.172970 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:57.172934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:50:58.857067 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.857044 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:58.970633 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970551 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-certs\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970633 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970588 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-envoy\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970633 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970609 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz8ns\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-kube-api-access-mz8ns\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970882 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970644 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-socket\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970882 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970663 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-data\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970882 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970682 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-credential-socket\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970882 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970699 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-istio-token\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970882 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970720 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ced37791-c686-4b65-91d7-631a375ca538-istio-podinfo\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.970882 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970743 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ced37791-c686-4b65-91d7-631a375ca538-istiod-ca-cert\") pod \"ced37791-c686-4b65-91d7-631a375ca538\" (UID: \"ced37791-c686-4b65-91d7-631a375ca538\") " Jul 22 16:50:58.971212 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970902 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:58.971212 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970956 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:58.971212 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.970967 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:58.971372 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.971269 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ced37791-c686-4b65-91d7-631a375ca538-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:50:58.971850 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.971827 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-data" (OuterVolumeSpecName: "istio-data") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:58.972992 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.972964 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-kube-api-access-mz8ns" (OuterVolumeSpecName: "kube-api-access-mz8ns") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "kube-api-access-mz8ns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:58.973128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.973063 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:50:58.973356 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.973340 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-istio-token" (OuterVolumeSpecName: "istio-token") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:58.973402 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:58.973373 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ced37791-c686-4b65-91d7-631a375ca538-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ced37791-c686-4b65-91d7-631a375ca538" (UID: "ced37791-c686-4b65-91d7-631a375ca538"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 16:50:59.072048 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072016 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072048 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072041 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072048 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072050 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz8ns\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-kube-api-access-mz8ns\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072061 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072069 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072077 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ced37791-c686-4b65-91d7-631a375ca538-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072101 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ced37791-c686-4b65-91d7-631a375ca538-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072109 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ced37791-c686-4b65-91d7-631a375ca538-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.072291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.072116 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ced37791-c686-4b65-91d7-631a375ca538-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:50:59.105063 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.105030 2555 generic.go:356] "Generic (PLEG): container finished" podID="ced37791-c686-4b65-91d7-631a375ca538" containerID="31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd" exitCode=0 Jul 22 16:50:59.105210 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.105082 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" Jul 22 16:50:59.105210 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.105077 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" event={"ID":"ced37791-c686-4b65-91d7-631a375ca538","Type":"ContainerDied","Data":"31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd"} Jul 22 16:50:59.105210 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.105181 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7" event={"ID":"ced37791-c686-4b65-91d7-631a375ca538","Type":"ContainerDied","Data":"9f36c5d720e38e19c447d29a2f358aa8329e37ba5add64cfb3e603d40ef8809e"} Jul 22 16:50:59.105210 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.105204 2555 scope.go:117] "RemoveContainer" containerID="31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd" Jul 22 16:50:59.114190 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.114173 2555 scope.go:117] "RemoveContainer" containerID="21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2" Jul 22 16:50:59.121791 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.121776 2555 scope.go:117] "RemoveContainer" containerID="31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd" Jul 22 16:50:59.122049 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:50:59.122031 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd\": container with ID starting with 31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd not found: ID does not exist" containerID="31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd" Jul 22 16:50:59.122134 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.122057 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd"} err="failed to get container status \"31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd\": rpc error: code = NotFound desc = could not find container \"31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd\": container with ID starting with 31edbe5cffb875a2dada4e6802c4dff01b39cc0407677c49cdd69386b54dedbd not found: ID does not exist" Jul 22 16:50:59.122134 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.122075 2555 scope.go:117] "RemoveContainer" containerID="21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2" Jul 22 16:50:59.122347 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:50:59.122328 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2\": container with ID starting with 21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2 not found: ID does not exist" containerID="21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2" Jul 22 16:50:59.122400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.122356 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2"} err="failed to get container status \"21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2\": rpc error: code = NotFound desc = could not find container \"21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2\": container with ID starting with 21764d95460c62f014414391817d96123b6b64fb68cc941b664df1e2cf9ae1a2 not found: ID does not exist" Jul 22 16:50:59.125615 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.125594 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7"] Jul 22 16:50:59.127604 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.127585 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c45d36b4-gw-openshift-default-67ff8ffd99-ngvr7"] Jul 22 16:50:59.743521 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:50:59.743479 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ced37791-c686-4b65-91d7-631a375ca538" path="/var/lib/kubelet/pods/ced37791-c686-4b65-91d7-631a375ca538/volumes" Jul 22 16:51:03.089259 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:03.089225 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:51:11.167429 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:11.166253 2555 generic.go:356] "Generic (PLEG): container finished" podID="7805e5de-e203-4110-b7a0-09117de54fe8" containerID="b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7" exitCode=0 Jul 22 16:51:11.167429 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:11.166360 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" event={"ID":"7805e5de-e203-4110-b7a0-09117de54fe8","Type":"ContainerDied","Data":"b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7"} Jul 22 16:51:11.167429 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:11.166774 2555 scope.go:117] "RemoveContainer" containerID="b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7" Jul 22 16:51:11.169052 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:11.169026 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:11.169138 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:11.169110 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:11.169179 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:11.169144 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:12.096152 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:12.096120 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:51:12.172597 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:12.172563 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" event={"ID":"7805e5de-e203-4110-b7a0-09117de54fe8","Type":"ContainerStarted","Data":"29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172"} Jul 22 16:51:12.828059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:12.828020 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:51:12.828268 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:12.828196 2555 patch_prober.go:28] interesting pod/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 22 16:51:12.828333 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:12.828281 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 22 16:51:13.828016 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:13.827966 2555 patch_prober.go:28] interesting pod/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 22 16:51:13.828455 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:13.828035 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 22 16:51:14.828005 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:14.827952 2555 patch_prober.go:28] interesting pod/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 22 16:51:14.828220 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:14.828026 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 22 16:51:15.827823 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:15.827785 2555 patch_prober.go:28] interesting pod/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 22 16:51:15.828150 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:15.828106 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 22 16:51:16.831930 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:16.831896 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:51:16.832446 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:16.832321 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:51:16.833062 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:16.833041 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:51:22.366351 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.366299 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7"] Jul 22 16:51:22.366824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.366703 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" Jul 22 16:51:22.366824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.366721 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" Jul 22 16:51:22.366824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.366736 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" Jul 22 16:51:22.366824 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.366747 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" Jul 22 16:51:22.367029 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.366827 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" Jul 22 16:51:22.367029 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.366839 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ced37791-c686-4b65-91d7-631a375ca538" containerName="istio-proxy" Jul 22 16:51:22.370325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.370301 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" Jul 22 16:51:22.372698 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.372678 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sh5d4\"" Jul 22 16:51:22.372803 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.372706 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 16:51:22.376300 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.376275 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7"] Jul 22 16:51:22.472263 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.472230 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c4ead4-5680-40ae-b765-e130e3c50fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3000a004-fk4f7\" (UID: \"10c4ead4-5680-40ae-b765-e130e3c50fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" Jul 22 16:51:22.573642 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.573608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c4ead4-5680-40ae-b765-e130e3c50fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3000a004-fk4f7\" (UID: \"10c4ead4-5680-40ae-b765-e130e3c50fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" Jul 22 16:51:22.574223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.574205 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c4ead4-5680-40ae-b765-e130e3c50fc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3000a004-fk4f7\" (UID: \"10c4ead4-5680-40ae-b765-e130e3c50fc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" Jul 22 16:51:22.684451 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.684349 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" Jul 22 16:51:22.811827 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:22.811798 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7"] Jul 22 16:51:22.814118 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:51:22.814075 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c4ead4_5680_40ae_b765_e130e3c50fc9.slice/crio-58c7a0ad3b78c6bdf0c226c3d2a97b7197349c635f6ad5daf26abc2a6ebde79f WatchSource:0}: Error finding container 58c7a0ad3b78c6bdf0c226c3d2a97b7197349c635f6ad5daf26abc2a6ebde79f: Status 404 returned error can't find the container with id 58c7a0ad3b78c6bdf0c226c3d2a97b7197349c635f6ad5daf26abc2a6ebde79f Jul 22 16:51:23.217296 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:23.217259 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" event={"ID":"10c4ead4-5680-40ae-b765-e130e3c50fc9","Type":"ContainerStarted","Data":"e2ff47993838f271f2e5d41d1094203d6625aa92f1822bc1b6546d3a96986825"} Jul 22 16:51:23.217487 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:23.217307 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" event={"ID":"10c4ead4-5680-40ae-b765-e130e3c50fc9","Type":"ContainerStarted","Data":"58c7a0ad3b78c6bdf0c226c3d2a97b7197349c635f6ad5daf26abc2a6ebde79f"} Jul 22 16:51:23.233232 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:23.233184 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" podStartSLOduration=1.233167318 podStartE2EDuration="1.233167318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:51:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:51:23.231953583 +0000 UTC m=+1725.937396126" watchObservedRunningTime="2026-07-22 16:51:23.233167318 +0000 UTC m=+1725.938609851" Jul 22 16:51:24.222400 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:24.222346 2555 generic.go:356] "Generic (PLEG): container finished" podID="10c4ead4-5680-40ae-b765-e130e3c50fc9" containerID="e2ff47993838f271f2e5d41d1094203d6625aa92f1822bc1b6546d3a96986825" exitCode=0 Jul 22 16:51:24.222778 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:24.222431 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" event={"ID":"10c4ead4-5680-40ae-b765-e130e3c50fc9","Type":"ContainerDied","Data":"e2ff47993838f271f2e5d41d1094203d6625aa92f1822bc1b6546d3a96986825"} Jul 22 16:51:25.350424 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:25.350399 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" Jul 22 16:51:25.495227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:25.495131 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c4ead4-5680-40ae-b765-e130e3c50fc9-maas-api-service-ca\") pod \"10c4ead4-5680-40ae-b765-e130e3c50fc9\" (UID: \"10c4ead4-5680-40ae-b765-e130e3c50fc9\") " Jul 22 16:51:25.495478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:25.495455 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c4ead4-5680-40ae-b765-e130e3c50fc9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "10c4ead4-5680-40ae-b765-e130e3c50fc9" (UID: "10c4ead4-5680-40ae-b765-e130e3c50fc9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:51:25.596729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:25.596679 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c4ead4-5680-40ae-b765-e130e3c50fc9-maas-api-service-ca\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:26.231493 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:26.231451 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" event={"ID":"10c4ead4-5680-40ae-b765-e130e3c50fc9","Type":"ContainerDied","Data":"58c7a0ad3b78c6bdf0c226c3d2a97b7197349c635f6ad5daf26abc2a6ebde79f"} Jul 22 16:51:26.231493 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:26.231470 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7" Jul 22 16:51:26.231493 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:26.231498 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58c7a0ad3b78c6bdf0c226c3d2a97b7197349c635f6ad5daf26abc2a6ebde79f" Jul 22 16:51:32.914172 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:32.914133 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:35.912894 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:35.912859 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:51:37.442783 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:37.442744 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7"] Jul 22 16:51:37.443270 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:37.442798 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:37.445203 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:37.445178 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3000a004-fk4f7"] Jul 22 16:51:37.748578 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:37.748502 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10c4ead4-5680-40ae-b765-e130e3c50fc9" path="/var/lib/kubelet/pods/10c4ead4-5680-40ae-b765-e130e3c50fc9/volumes" Jul 22 16:51:38.679493 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:38.679457 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7"] Jul 22 16:51:38.679906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:38.679715 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" containerID="cri-o://29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172" gracePeriod=30 Jul 22 16:51:38.702888 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:38.702832 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:38.970346 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:38.970260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:39.217574 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.217533 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:39.514429 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.514382 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb"] Jul 22 16:51:39.514803 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.514786 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c4ead4-5680-40ae-b765-e130e3c50fc9" containerName="revoke-keys" Jul 22 16:51:39.514859 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.514806 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c4ead4-5680-40ae-b765-e130e3c50fc9" containerName="revoke-keys" Jul 22 16:51:39.514926 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.514915 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c4ead4-5680-40ae-b765-e130e3c50fc9" containerName="revoke-keys" Jul 22 16:51:39.518006 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.517984 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.520588 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.520568 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-57182c73-openshift-default-dockercfg-wvs52\"" Jul 22 16:51:39.530260 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.530225 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb"] Jul 22 16:51:39.549113 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.549049 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:39.603457 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603417 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-socket\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603642 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603469 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b8bf27-f06e-4458-8bea-588a1d00f02f-istiod-ca-cert\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603642 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603539 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-token\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603642 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603571 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-data\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603642 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603596 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-envoy\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603642 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603612 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-certs\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603913 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-podinfo\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603913 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603735 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-credential-socket\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.603913 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.603753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt6xx\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-kube-api-access-gt6xx\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.704575 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704536 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-data\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.704575 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704577 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-envoy\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704596 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-certs\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704623 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-podinfo\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704648 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-credential-socket\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt6xx\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-kube-api-access-gt6xx\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704809 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-socket\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704851 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b8bf27-f06e-4458-8bea-588a1d00f02f-istiod-ca-cert\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705074 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.704890 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-token\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705407 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.705118 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-data\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705407 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.705150 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-certs\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705407 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.705393 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-socket\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705672 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.705649 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-credential-socket\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.705864 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.705842 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b8bf27-f06e-4458-8bea-588a1d00f02f-istiod-ca-cert\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.707223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.707205 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-envoy\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.707473 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.707453 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-podinfo\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.712538 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.712518 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-token\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.712675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.712652 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt6xx\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-kube-api-access-gt6xx\") pod \"e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.828631 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.828599 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:39.945134 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.945068 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:39.965454 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.965426 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb"] Jul 22 16:51:39.967818 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:51:39.967789 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b8bf27_f06e_4458_8bea_588a1d00f02f.slice/crio-3a046afc29636f3ad0769d4aca31f39ad3490d917a74c028aaac6b78624dba81 WatchSource:0}: Error finding container 3a046afc29636f3ad0769d4aca31f39ad3490d917a74c028aaac6b78624dba81: Status 404 returned error can't find the container with id 3a046afc29636f3ad0769d4aca31f39ad3490d917a74c028aaac6b78624dba81 Jul 22 16:51:39.970020 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.969991 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:39.970112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.970055 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:39.970112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:39.970107 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:40.290357 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:40.290317 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" event={"ID":"57b8bf27-f06e-4458-8bea-588a1d00f02f","Type":"ContainerStarted","Data":"d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e"} Jul 22 16:51:40.290533 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:40.290380 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" event={"ID":"57b8bf27-f06e-4458-8bea-588a1d00f02f","Type":"ContainerStarted","Data":"3a046afc29636f3ad0769d4aca31f39ad3490d917a74c028aaac6b78624dba81"} Jul 22 16:51:40.309240 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:40.309173 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" podStartSLOduration=1.3091522420000001 podStartE2EDuration="1.309152242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:51:40.30901973 +0000 UTC m=+1743.014462274" watchObservedRunningTime="2026-07-22 16:51:40.309152242 +0000 UTC m=+1743.014594795" Jul 22 16:51:40.829016 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:40.828986 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:41.655034 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:41.654968 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:41.731643 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:41.731604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:41.798887 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:41.798848 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:41.852649 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:41.852618 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:42.012472 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:42.012379 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:42.217605 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:42.217562 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:42.298712 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:42.298682 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:42.299792 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:42.299776 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:42.324061 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:42.324013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:43.922179 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.922155 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:51:43.940930 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.940907 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-envoy\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.940935 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-data\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.940955 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5pdf\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-kube-api-access-f5pdf\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.940985 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7805e5de-e203-4110-b7a0-09117de54fe8-istio-podinfo\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941002 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-socket\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941059 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941051 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-credential-socket\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941352 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941067 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-istio-token\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941352 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941103 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7805e5de-e203-4110-b7a0-09117de54fe8-istiod-ca-cert\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941352 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941149 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-certs\") pod \"7805e5de-e203-4110-b7a0-09117de54fe8\" (UID: \"7805e5de-e203-4110-b7a0-09117de54fe8\") " Jul 22 16:51:43.941589 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941539 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:51:43.941701 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941674 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:51:43.941935 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941836 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7805e5de-e203-4110-b7a0-09117de54fe8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:51:43.941935 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941877 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:51:43.942053 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.941933 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-data" (OuterVolumeSpecName: "istio-data") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:51:43.943706 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.943598 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7805e5de-e203-4110-b7a0-09117de54fe8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 16:51:43.944402 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.944131 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-kube-api-access-f5pdf" (OuterVolumeSpecName: "kube-api-access-f5pdf") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "kube-api-access-f5pdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:51:43.946764 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.946740 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:51:43.946764 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:43.946759 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-istio-token" (OuterVolumeSpecName: "istio-token") pod "7805e5de-e203-4110-b7a0-09117de54fe8" (UID: "7805e5de-e203-4110-b7a0-09117de54fe8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:51:44.042737 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042702 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042737 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042733 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042746 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5pdf\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-kube-api-access-f5pdf\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042758 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7805e5de-e203-4110-b7a0-09117de54fe8-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042769 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042780 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042791 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7805e5de-e203-4110-b7a0-09117de54fe8-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042802 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7805e5de-e203-4110-b7a0-09117de54fe8-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.042947 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.042813 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7805e5de-e203-4110-b7a0-09117de54fe8-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:51:44.308741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.308703 2555 generic.go:356] "Generic (PLEG): container finished" podID="7805e5de-e203-4110-b7a0-09117de54fe8" containerID="29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172" exitCode=0 Jul 22 16:51:44.308915 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.308760 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" Jul 22 16:51:44.308915 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.308788 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" event={"ID":"7805e5de-e203-4110-b7a0-09117de54fe8","Type":"ContainerDied","Data":"29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172"} Jul 22 16:51:44.308915 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.308855 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7" event={"ID":"7805e5de-e203-4110-b7a0-09117de54fe8","Type":"ContainerDied","Data":"f0012c4d9e8628c351f6dd5f461a325ab1d780f467652696395768d04618e75d"} Jul 22 16:51:44.308915 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.308873 2555 scope.go:117] "RemoveContainer" containerID="29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172" Jul 22 16:51:44.318320 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.318301 2555 scope.go:117] "RemoveContainer" containerID="b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7" Jul 22 16:51:44.326017 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.326000 2555 scope.go:117] "RemoveContainer" containerID="29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172" Jul 22 16:51:44.326294 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:51:44.326276 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172\": container with ID starting with 29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172 not found: ID does not exist" containerID="29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172" Jul 22 16:51:44.326345 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.326302 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172"} err="failed to get container status \"29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172\": rpc error: code = NotFound desc = could not find container \"29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172\": container with ID starting with 29b4d6ddb557d60f74cbc31025565b9dc19f49e3c6466c5e80d4f05b96190172 not found: ID does not exist" Jul 22 16:51:44.326345 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.326323 2555 scope.go:117] "RemoveContainer" containerID="b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7" Jul 22 16:51:44.326558 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:51:44.326541 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7\": container with ID starting with b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7 not found: ID does not exist" containerID="b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7" Jul 22 16:51:44.326600 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.326565 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7"} err="failed to get container status \"b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7\": rpc error: code = NotFound desc = could not find container \"b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7\": container with ID starting with b55a2d4ad59efae9bf512d2e9060037da72c3d55722a6f02409e800f8f21fcc7 not found: ID does not exist" Jul 22 16:51:44.330549 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.330525 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7"] Jul 22 16:51:44.334638 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:44.334617 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3000a004-openshift-default-f68b7f5f5-5hgq7"] Jul 22 16:51:45.743297 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:45.743264 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" path="/var/lib/kubelet/pods/7805e5de-e203-4110-b7a0-09117de54fe8/volumes" Jul 22 16:51:47.100106 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:47.100051 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:51:51.121997 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:51.121951 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:51.213754 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:51.213714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:51:55.361486 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:55.361404 2555 generic.go:356] "Generic (PLEG): container finished" podID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerID="d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e" exitCode=0 Jul 22 16:51:55.361486 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:55.361447 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" event={"ID":"57b8bf27-f06e-4458-8bea-588a1d00f02f","Type":"ContainerDied","Data":"d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e"} Jul 22 16:51:55.361938 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:55.361809 2555 scope.go:117] "RemoveContainer" containerID="d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e" Jul 22 16:51:55.362784 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:55.362748 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:55.362880 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:55.362827 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:55.362880 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:55.362858 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:51:56.366910 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:56.366877 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" event={"ID":"57b8bf27-f06e-4458-8bea-588a1d00f02f","Type":"ContainerStarted","Data":"b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d"} Jul 22 16:51:56.829311 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:56.829268 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:56.830445 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:56.830415 2555 patch_prober.go:28] interesting pod/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 22 16:51:56.830579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:56.830467 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 22 16:51:57.298791 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:57.298754 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:51:57.828987 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:57.828938 2555 patch_prober.go:28] interesting pod/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 22 16:51:57.829523 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:57.829004 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 22 16:51:58.829410 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:58.829368 2555 patch_prober.go:28] interesting pod/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 22 16:51:58.829879 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:58.829431 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 22 16:51:59.829492 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:59.829448 2555 patch_prober.go:28] interesting pod/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 22 16:51:59.829880 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:51:59.829509 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 22 16:52:00.833992 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:00.833960 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:52:00.835065 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:00.835041 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:52:02.978835 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:02.978769 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:03.080013 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:03.079978 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:06.606215 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.606176 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7"] Jul 22 16:52:06.606680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.606504 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" Jul 22 16:52:06.606680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.606516 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" Jul 22 16:52:06.606680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.606524 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" Jul 22 16:52:06.606680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.606530 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" Jul 22 16:52:06.606680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.606596 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" Jul 22 16:52:06.609694 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.609678 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" Jul 22 16:52:06.612208 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.612186 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sh5d4\"" Jul 22 16:52:06.612331 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.612187 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 16:52:06.615729 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.615709 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7"] Jul 22 16:52:06.743655 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.743619 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86641828-35b6-4bf0-93ad-19124e3f6157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57182c73-wjbf7\" (UID: \"86641828-35b6-4bf0-93ad-19124e3f6157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" Jul 22 16:52:06.844115 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.844058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86641828-35b6-4bf0-93ad-19124e3f6157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57182c73-wjbf7\" (UID: \"86641828-35b6-4bf0-93ad-19124e3f6157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" Jul 22 16:52:06.844641 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.844620 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86641828-35b6-4bf0-93ad-19124e3f6157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57182c73-wjbf7\" (UID: \"86641828-35b6-4bf0-93ad-19124e3f6157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" Jul 22 16:52:06.921894 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:06.921806 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" Jul 22 16:52:07.047510 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:07.047485 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7"] Jul 22 16:52:07.050104 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:52:07.050061 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86641828_35b6_4bf0_93ad_19124e3f6157.slice/crio-03763a4ad590fd974362795cb5d47cd10373e6539e50e791b8a618ef85897894 WatchSource:0}: Error finding container 03763a4ad590fd974362795cb5d47cd10373e6539e50e791b8a618ef85897894: Status 404 returned error can't find the container with id 03763a4ad590fd974362795cb5d47cd10373e6539e50e791b8a618ef85897894 Jul 22 16:52:07.417635 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:07.417603 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" event={"ID":"86641828-35b6-4bf0-93ad-19124e3f6157","Type":"ContainerStarted","Data":"b9436aa4fcef8c46cbfdbbe86e21faa3012566ce0f96586c16dbd7064bddfb56"} Jul 22 16:52:07.417635 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:07.417652 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" event={"ID":"86641828-35b6-4bf0-93ad-19124e3f6157","Type":"ContainerStarted","Data":"03763a4ad590fd974362795cb5d47cd10373e6539e50e791b8a618ef85897894"} Jul 22 16:52:07.433353 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:07.433295 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" podStartSLOduration=1.433275596 podStartE2EDuration="1.433275596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:52:07.432708297 +0000 UTC m=+1770.138150838" watchObservedRunningTime="2026-07-22 16:52:07.433275596 +0000 UTC m=+1770.138718141" Jul 22 16:52:08.423579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:08.423547 2555 generic.go:356] "Generic (PLEG): container finished" podID="86641828-35b6-4bf0-93ad-19124e3f6157" containerID="b9436aa4fcef8c46cbfdbbe86e21faa3012566ce0f96586c16dbd7064bddfb56" exitCode=0 Jul 22 16:52:08.423579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:08.423586 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" event={"ID":"86641828-35b6-4bf0-93ad-19124e3f6157","Type":"ContainerDied","Data":"b9436aa4fcef8c46cbfdbbe86e21faa3012566ce0f96586c16dbd7064bddfb56"} Jul 22 16:52:09.552152 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:09.552130 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" Jul 22 16:52:09.666440 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:09.666408 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86641828-35b6-4bf0-93ad-19124e3f6157-maas-api-service-ca\") pod \"86641828-35b6-4bf0-93ad-19124e3f6157\" (UID: \"86641828-35b6-4bf0-93ad-19124e3f6157\") " Jul 22 16:52:09.671143 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:09.667218 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86641828-35b6-4bf0-93ad-19124e3f6157-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86641828-35b6-4bf0-93ad-19124e3f6157" (UID: "86641828-35b6-4bf0-93ad-19124e3f6157"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:52:09.768304 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:09.768277 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86641828-35b6-4bf0-93ad-19124e3f6157-maas-api-service-ca\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:10.432684 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:10.432649 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" event={"ID":"86641828-35b6-4bf0-93ad-19124e3f6157","Type":"ContainerDied","Data":"03763a4ad590fd974362795cb5d47cd10373e6539e50e791b8a618ef85897894"} Jul 22 16:52:10.432684 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:10.432683 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7" Jul 22 16:52:10.432890 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:10.432710 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03763a4ad590fd974362795cb5d47cd10373e6539e50e791b8a618ef85897894" Jul 22 16:52:17.218334 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:17.218292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:20.497234 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:20.497200 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:52:21.677004 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:21.676966 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7"] Jul 22 16:52:21.680164 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:21.680137 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57182c73-wjbf7"] Jul 22 16:52:21.680351 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:21.680329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:21.743688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:21.743652 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86641828-35b6-4bf0-93ad-19124e3f6157" path="/var/lib/kubelet/pods/86641828-35b6-4bf0-93ad-19124e3f6157/volumes" Jul 22 16:52:21.815937 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:21.815904 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb"] Jul 22 16:52:21.816180 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:21.816140 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" containerID="cri-o://b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d" gracePeriod=30 Jul 22 16:52:21.832279 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:21.832246 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:22.082697 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:22.082664 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:22.564057 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:22.564024 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:22.662774 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:22.662733 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:22.759545 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:22.759511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:24.204030 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:24.203977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:27.056256 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.056231 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:52:27.210767 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210673 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-certs\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.210767 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210710 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt6xx\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-kube-api-access-gt6xx\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.210767 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210746 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b8bf27-f06e-4458-8bea-588a1d00f02f-istiod-ca-cert\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.210767 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210761 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-data\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.211135 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210785 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-token\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.211135 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210804 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-credential-socket\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.211135 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210901 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-podinfo\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.211135 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210937 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-socket\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.211135 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.210968 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-envoy\") pod \"57b8bf27-f06e-4458-8bea-588a1d00f02f\" (UID: \"57b8bf27-f06e-4458-8bea-588a1d00f02f\") " Jul 22 16:52:27.211135 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.211048 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:52:27.211415 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.211238 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57b8bf27-f06e-4458-8bea-588a1d00f02f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:52:27.211415 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.211265 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:52:27.211415 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.211300 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.211415 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.211391 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:52:27.211790 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.211758 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-data" (OuterVolumeSpecName: "istio-data") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:52:27.213166 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.213135 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-token" (OuterVolumeSpecName: "istio-token") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:52:27.213241 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.213166 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-kube-api-access-gt6xx" (OuterVolumeSpecName: "kube-api-access-gt6xx") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "kube-api-access-gt6xx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:52:27.213703 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.213687 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:52:27.213746 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.213719 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57b8bf27-f06e-4458-8bea-588a1d00f02f" (UID: "57b8bf27-f06e-4458-8bea-588a1d00f02f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 16:52:27.311702 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311665 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57b8bf27-f06e-4458-8bea-588a1d00f02f-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.311702 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311697 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.311702 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311705 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.311936 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311715 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.311936 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311725 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.311936 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311733 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.311936 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311741 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57b8bf27-f06e-4458-8bea-588a1d00f02f-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.311936 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.311749 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt6xx\" (UniqueName: \"kubernetes.io/projected/57b8bf27-f06e-4458-8bea-588a1d00f02f-kube-api-access-gt6xx\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:52:27.506377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.506287 2555 generic.go:356] "Generic (PLEG): container finished" podID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerID="b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d" exitCode=0 Jul 22 16:52:27.506377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.506337 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" Jul 22 16:52:27.506377 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.506350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" event={"ID":"57b8bf27-f06e-4458-8bea-588a1d00f02f","Type":"ContainerDied","Data":"b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d"} Jul 22 16:52:27.506629 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.506403 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb" event={"ID":"57b8bf27-f06e-4458-8bea-588a1d00f02f","Type":"ContainerDied","Data":"3a046afc29636f3ad0769d4aca31f39ad3490d917a74c028aaac6b78624dba81"} Jul 22 16:52:27.506629 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.506432 2555 scope.go:117] "RemoveContainer" containerID="b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d" Jul 22 16:52:27.515290 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.515274 2555 scope.go:117] "RemoveContainer" containerID="d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e" Jul 22 16:52:27.522840 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.522822 2555 scope.go:117] "RemoveContainer" containerID="b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d" Jul 22 16:52:27.523077 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:52:27.523056 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d\": container with ID starting with b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d not found: ID does not exist" containerID="b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d" Jul 22 16:52:27.523192 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.523172 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d"} err="failed to get container status \"b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d\": rpc error: code = NotFound desc = could not find container \"b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d\": container with ID starting with b11fb52c5aa3bde64242d513051d1449ea7d5695d964c610cbfa807a5197ce0d not found: ID does not exist" Jul 22 16:52:27.523244 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.523195 2555 scope.go:117] "RemoveContainer" containerID="d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e" Jul 22 16:52:27.523441 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:52:27.523423 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e\": container with ID starting with d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e not found: ID does not exist" containerID="d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e" Jul 22 16:52:27.523499 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.523446 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e"} err="failed to get container status \"d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e\": rpc error: code = NotFound desc = could not find container \"d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e\": container with ID starting with d414b860b50b39341a286fde98cd982a512b77f160484f24e81ebb555bdc361e not found: ID does not exist" Jul 22 16:52:27.527674 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.527655 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb"] Jul 22 16:52:27.534587 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.534566 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-57182c73-openshift-default-66f68c8cd7-r99jb"] Jul 22 16:52:27.743025 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:27.742993 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" path="/var/lib/kubelet/pods/57b8bf27-f06e-4458-8bea-588a1d00f02f/volumes" Jul 22 16:52:30.101773 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:30.101679 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:30.161963 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:30.161929 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:30.247292 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:30.247257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:30.345478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:30.345443 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:30.446140 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:30.446035 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:30.800404 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:30.800379 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:52:56.781898 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:52:56.781858 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:03.426100 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:03.426040 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:03.527817 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:03.527782 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:03.832598 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:03.832558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:19.604915 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:19.604879 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:25.185936 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:25.185900 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:25.290155 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:25.290110 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:25.386966 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:25.386931 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:35.967982 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:35.967945 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:36.845201 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:36.845163 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:43.260557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:43.260504 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:43.360906 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:43.360869 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:43.463080 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:43.463047 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:43.563916 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:43.563877 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:44.337216 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:44.337181 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:44.432999 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:44.432961 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:44.533595 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:44.533558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:47.689866 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:47.689828 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:53:51.358245 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:51.358194 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:55.193852 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:55.193817 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:53:57.769756 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:57.769718 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:57.871512 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:57.871477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:53:58.609612 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:53:58.609530 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:04.920148 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:04.920111 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:05.009011 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.008975 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:54:05.011931 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.011903 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:05.173431 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.173341 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:05.513270 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.513189 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6"] Jul 22 16:54:05.513832 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.513811 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86641828-35b6-4bf0-93ad-19124e3f6157" containerName="revoke-keys" Jul 22 16:54:05.513966 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.513955 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="86641828-35b6-4bf0-93ad-19124e3f6157" containerName="revoke-keys" Jul 22 16:54:05.514047 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.514037 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" Jul 22 16:54:05.514152 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.514142 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" Jul 22 16:54:05.514359 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.514347 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" Jul 22 16:54:05.514443 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.514434 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="86641828-35b6-4bf0-93ad-19124e3f6157" containerName="revoke-keys" Jul 22 16:54:05.514510 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.514502 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7805e5de-e203-4110-b7a0-09117de54fe8" containerName="istio-proxy" Jul 22 16:54:05.514579 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.514571 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" Jul 22 16:54:05.517960 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.517939 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.521054 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.521032 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-87fbb90e-openshift-default-dockercfg-lncmz\"" Jul 22 16:54:05.535139 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.535114 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6"] Jul 22 16:54:05.542207 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.542171 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:05.602991 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.602952 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-certs\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603177 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603010 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istiod-ca-cert\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603177 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-podinfo\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603177 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603114 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-socket\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603338 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603231 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-token\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603338 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603304 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-envoy\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603366 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-credential-socket\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603396 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv7r6\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-kube-api-access-jv7r6\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.603478 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.603429 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-data\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.653740 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.653704 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:05.703857 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.703820 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-certs\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.703857 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.703863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istiod-ca-cert\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.703883 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-podinfo\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.703903 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-socket\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.703924 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-token\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.703979 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-envoy\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704010 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-credential-socket\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704157 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704031 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv7r6\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-kube-api-access-jv7r6\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704508 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-data\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704508 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704318 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-certs\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704508 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704484 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-credential-socket\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704664 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704557 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-data\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704664 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704567 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-socket\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.704741 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.704706 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istiod-ca-cert\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.706417 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.706394 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-envoy\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.706712 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.706691 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-podinfo\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.712339 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.712318 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-token\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.712601 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.712579 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv7r6\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-kube-api-access-jv7r6\") pod \"e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.828877 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.828837 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:05.969681 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.969655 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6"] Jul 22 16:54:05.972287 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:54:05.972257 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1518db_04e1_4af2_a59a_a8a8e7de5f5c.slice/crio-33c0587423e15163f92dcd3da76a4b968882c1e112e5cf5e00f4e37fb276b148 WatchSource:0}: Error finding container 33c0587423e15163f92dcd3da76a4b968882c1e112e5cf5e00f4e37fb276b148: Status 404 returned error can't find the container with id 33c0587423e15163f92dcd3da76a4b968882c1e112e5cf5e00f4e37fb276b148 Jul 22 16:54:05.974388 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.974350 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:54:05.974472 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.974421 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:54:05.974472 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:05.974449 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:54:06.917207 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:06.917157 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" event={"ID":"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c","Type":"ContainerStarted","Data":"8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b"} Jul 22 16:54:06.917384 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:06.917250 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" event={"ID":"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c","Type":"ContainerStarted","Data":"33c0587423e15163f92dcd3da76a4b968882c1e112e5cf5e00f4e37fb276b148"} Jul 22 16:54:06.940000 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:06.939944 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" podStartSLOduration=1.9399170300000002 podStartE2EDuration="1.93991703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:54:06.937688024 +0000 UTC m=+1889.643130566" watchObservedRunningTime="2026-07-22 16:54:06.93991703 +0000 UTC m=+1889.645359552" Jul 22 16:54:07.829149 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:07.829119 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:07.833910 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:07.833884 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:07.921713 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:07.921681 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:07.922619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:07.922598 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:08.915617 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:08.915581 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:09.013040 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:09.013002 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:09.127370 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:09.127330 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:09.372650 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:09.372617 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:09.571716 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:09.571681 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:09.672678 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:09.672581 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:13.431243 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:13.431201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:14.188954 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:14.188911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:14.289284 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:14.289248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:14.589323 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:14.589288 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:18.295953 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:18.295911 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:54:19.979245 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:19.979158 2555 generic.go:356] "Generic (PLEG): container finished" podID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerID="8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b" exitCode=0 Jul 22 16:54:19.979245 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:19.979235 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" event={"ID":"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c","Type":"ContainerDied","Data":"8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b"} Jul 22 16:54:19.979666 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:19.979637 2555 scope.go:117] "RemoveContainer" containerID="8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b" Jul 22 16:54:19.980619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:19.980595 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:54:19.980681 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:19.980668 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:54:19.980727 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:19.980720 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:54:20.491449 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.491413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:20.593557 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.593520 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:20.867838 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.867801 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x"] Jul 22 16:54:20.868282 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.868252 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" Jul 22 16:54:20.868282 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.868279 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="57b8bf27-f06e-4458-8bea-588a1d00f02f" containerName="istio-proxy" Jul 22 16:54:20.872458 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.872424 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" Jul 22 16:54:20.875885 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.875423 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sh5d4\"" Jul 22 16:54:20.875885 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.875731 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 16:54:20.879380 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.878641 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x"] Jul 22 16:54:20.985901 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:20.985864 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" event={"ID":"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c","Type":"ContainerStarted","Data":"4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e"} Jul 22 16:54:21.039221 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.039171 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36e6a5-843e-4cc5-ab02-607dee9bde9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x\" (UID: \"2a36e6a5-843e-4cc5-ab02-607dee9bde9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" Jul 22 16:54:21.140472 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.140361 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36e6a5-843e-4cc5-ab02-607dee9bde9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x\" (UID: \"2a36e6a5-843e-4cc5-ab02-607dee9bde9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" Jul 22 16:54:21.141005 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.140977 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36e6a5-843e-4cc5-ab02-607dee9bde9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x\" (UID: \"2a36e6a5-843e-4cc5-ab02-607dee9bde9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" Jul 22 16:54:21.191894 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.191857 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" Jul 22 16:54:21.371225 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.371201 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x"] Jul 22 16:54:21.373340 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:54:21.373311 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a36e6a5_843e_4cc5_ab02_607dee9bde9b.slice/crio-8883392e812a8466c2bf5237e57e86612b3d766e434f73c1053b79686227bf95 WatchSource:0}: Error finding container 8883392e812a8466c2bf5237e57e86612b3d766e434f73c1053b79686227bf95: Status 404 returned error can't find the container with id 8883392e812a8466c2bf5237e57e86612b3d766e434f73c1053b79686227bf95 Jul 22 16:54:21.829325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.829281 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:21.830565 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.830539 2555 patch_prober.go:28] interesting pod/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 22 16:54:21.830695 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.830588 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 22 16:54:21.992629 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.992583 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" event={"ID":"2a36e6a5-843e-4cc5-ab02-607dee9bde9b","Type":"ContainerStarted","Data":"81675024e8e9d9410444c839910b78329a936d026291cbc0f7afd39e1671f03f"} Jul 22 16:54:21.993010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:21.992652 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" event={"ID":"2a36e6a5-843e-4cc5-ab02-607dee9bde9b","Type":"ContainerStarted","Data":"8883392e812a8466c2bf5237e57e86612b3d766e434f73c1053b79686227bf95"} Jul 22 16:54:22.009033 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:22.008753 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" podStartSLOduration=2.008732303 podStartE2EDuration="2.008732303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:54:22.006309655 +0000 UTC m=+1904.711752199" watchObservedRunningTime="2026-07-22 16:54:22.008732303 +0000 UTC m=+1904.714174846" Jul 22 16:54:22.829566 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:22.829522 2555 patch_prober.go:28] interesting pod/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 22 16:54:22.829816 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:22.829587 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 22 16:54:22.922019 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:22.921902 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:22.998016 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:22.997984 2555 generic.go:356] "Generic (PLEG): container finished" podID="2a36e6a5-843e-4cc5-ab02-607dee9bde9b" containerID="81675024e8e9d9410444c839910b78329a936d026291cbc0f7afd39e1671f03f" exitCode=0 Jul 22 16:54:22.998430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:22.998069 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" event={"ID":"2a36e6a5-843e-4cc5-ab02-607dee9bde9b","Type":"ContainerDied","Data":"81675024e8e9d9410444c839910b78329a936d026291cbc0f7afd39e1671f03f"} Jul 22 16:54:23.829497 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:23.829459 2555 patch_prober.go:28] interesting pod/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 22 16:54:23.829682 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:23.829515 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 22 16:54:24.161708 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:24.161681 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" Jul 22 16:54:24.267103 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:24.267061 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36e6a5-843e-4cc5-ab02-607dee9bde9b-maas-api-service-ca\") pod \"2a36e6a5-843e-4cc5-ab02-607dee9bde9b\" (UID: \"2a36e6a5-843e-4cc5-ab02-607dee9bde9b\") " Jul 22 16:54:24.267402 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:24.267375 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a36e6a5-843e-4cc5-ab02-607dee9bde9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a36e6a5-843e-4cc5-ab02-607dee9bde9b" (UID: "2a36e6a5-843e-4cc5-ab02-607dee9bde9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:54:24.368234 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:24.368189 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36e6a5-843e-4cc5-ab02-607dee9bde9b-maas-api-service-ca\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:24.829680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:24.829637 2555 patch_prober.go:28] interesting pod/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 22 16:54:24.829884 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:24.829699 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 22 16:54:25.007981 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:25.007939 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" event={"ID":"2a36e6a5-843e-4cc5-ab02-607dee9bde9b","Type":"ContainerDied","Data":"8883392e812a8466c2bf5237e57e86612b3d766e434f73c1053b79686227bf95"} Jul 22 16:54:25.007981 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:25.007965 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x" Jul 22 16:54:25.008214 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:25.008002 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8883392e812a8466c2bf5237e57e86612b3d766e434f73c1053b79686227bf95" Jul 22 16:54:25.833279 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:25.833249 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:25.834416 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:25.834390 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:30.235125 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:30.235072 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:31.384292 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:31.384252 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:34.193589 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:34.193550 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:54:35.922421 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:35.922386 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:35.927256 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:35.927228 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x"] Jul 22 16:54:35.929544 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:35.929520 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-87fbb90e-5sx5x"] Jul 22 16:54:36.318207 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:36.318168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:37.743271 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:37.743231 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a36e6a5-843e-4cc5-ab02-607dee9bde9b" path="/var/lib/kubelet/pods/2a36e6a5-843e-4cc5-ab02-607dee9bde9b/volumes" Jul 22 16:54:41.991550 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:41.991515 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:42.092204 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.092159 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:42.228201 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.228165 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6"] Jul 22 16:54:42.228403 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.228381 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" containerID="cri-o://4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e" gracePeriod=30 Jul 22 16:54:42.257229 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.257144 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:42.557672 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.557636 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:42.744999 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.744970 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:42.903756 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.903666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:42.966976 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:42.966941 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:47.479723 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.479696 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:47.552011 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.551981 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-certs\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552204 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552023 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-podinfo\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552204 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552041 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-token\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552204 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552056 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-credential-socket\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552204 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552072 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-data\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552334 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-envoy\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552389 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv7r6\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-kube-api-access-jv7r6\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552387 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:54:47.552430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552375 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:54:47.552430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552418 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istiod-ca-cert\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552680 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552501 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-socket\") pod \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\" (UID: \"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c\") " Jul 22 16:54:47.552821 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552751 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.552821 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552777 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.553023 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.552964 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:54:47.553235 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.553208 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:54:47.553308 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.553215 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:54:47.554443 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.554419 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 16:54:47.554550 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.554456 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:54:47.554550 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.554514 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:54:47.554629 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.554573 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-kube-api-access-jv7r6" (OuterVolumeSpecName: "kube-api-access-jv7r6") pod "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" (UID: "bc1518db-04e1-4af2-a59a-a8a8e7de5f5c"). InnerVolumeSpecName "kube-api-access-jv7r6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:54:47.653723 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.653695 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.653723 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.653721 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv7r6\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-kube-api-access-jv7r6\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.653977 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.653733 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.653977 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.653742 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.653977 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.653750 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.653977 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.653758 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:47.653977 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:47.653765 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:54:48.103553 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.103518 2555 generic.go:356] "Generic (PLEG): container finished" podID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerID="4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e" exitCode=0 Jul 22 16:54:48.103738 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.103565 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" event={"ID":"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c","Type":"ContainerDied","Data":"4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e"} Jul 22 16:54:48.103738 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.103582 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" Jul 22 16:54:48.103738 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.103614 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6" event={"ID":"bc1518db-04e1-4af2-a59a-a8a8e7de5f5c","Type":"ContainerDied","Data":"33c0587423e15163f92dcd3da76a4b968882c1e112e5cf5e00f4e37fb276b148"} Jul 22 16:54:48.103738 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.103642 2555 scope.go:117] "RemoveContainer" containerID="4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e" Jul 22 16:54:48.114600 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.114582 2555 scope.go:117] "RemoveContainer" containerID="8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b" Jul 22 16:54:48.122830 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.122810 2555 scope.go:117] "RemoveContainer" containerID="4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e" Jul 22 16:54:48.123140 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:54:48.123116 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e\": container with ID starting with 4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e not found: ID does not exist" containerID="4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e" Jul 22 16:54:48.123193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.123154 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e"} err="failed to get container status \"4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e\": rpc error: code = NotFound desc = could not find container \"4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e\": container with ID starting with 4b69a752ae867af2b615c5ba7bc82eed13af6b5cd4c292d1f02302b98640075e not found: ID does not exist" Jul 22 16:54:48.123193 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.123179 2555 scope.go:117] "RemoveContainer" containerID="8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b" Jul 22 16:54:48.123342 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.123324 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6"] Jul 22 16:54:48.123459 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:54:48.123443 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b\": container with ID starting with 8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b not found: ID does not exist" containerID="8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b" Jul 22 16:54:48.123503 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.123465 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b"} err="failed to get container status \"8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b\": rpc error: code = NotFound desc = could not find container \"8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b\": container with ID starting with 8545db7e588c454c7998874a66e230bd08ca4a52c8e2e426d9fc3071925e556b not found: ID does not exist" Jul 22 16:54:48.127024 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.127001 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-87fbb90e-openshift-default-6c9fcdcdc9-776s6"] Jul 22 16:54:48.735067 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.735035 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:48.843188 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.843134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:48.935499 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:48.935444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:49.119412 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:49.119374 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:49.744628 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:49.744597 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" path="/var/lib/kubelet/pods/bc1518db-04e1-4af2-a59a-a8a8e7de5f5c/volumes" Jul 22 16:54:54.095054 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:54.095007 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:54.190527 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:54.190490 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:54.752935 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:54.752895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:54:54.853757 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:54:54.853716 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:06.618627 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.618593 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:06.908585 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.908504 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p"] Jul 22 16:55:06.908869 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.908851 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" Jul 22 16:55:06.908978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.908871 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" Jul 22 16:55:06.908978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.908894 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" Jul 22 16:55:06.908978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.908902 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" Jul 22 16:55:06.908978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.908922 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a36e6a5-843e-4cc5-ab02-607dee9bde9b" containerName="revoke-keys" Jul 22 16:55:06.908978 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.908931 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a36e6a5-843e-4cc5-ab02-607dee9bde9b" containerName="revoke-keys" Jul 22 16:55:06.909272 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.909011 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" Jul 22 16:55:06.909272 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.909023 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a36e6a5-843e-4cc5-ab02-607dee9bde9b" containerName="revoke-keys" Jul 22 16:55:06.913508 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.913486 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:06.916164 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.916138 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b9fae547-openshift-default-dockercfg-2wtd4\"" Jul 22 16:55:06.922341 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.922315 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p"] Jul 22 16:55:06.929878 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:06.929841 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:07.009406 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009368 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-envoy\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009594 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009417 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-podinfo\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009594 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009447 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-data\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009594 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009554 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59548\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-kube-api-access-59548\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009733 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009603 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-socket\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009733 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009647 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-token\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009733 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009687 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-credential-socket\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009733 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009719 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istiod-ca-cert\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.009858 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.009768 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-certs\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.067142 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.067106 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:07.110599 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.110562 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istiod-ca-cert\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.110599 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.110603 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-certs\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.110838 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.110622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-envoy\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.110838 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.110746 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-podinfo\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.110855 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-data\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.110952 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59548\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-kube-api-access-59548\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.110994 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-certs\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.111005 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-socket\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.111291 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-data\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.111289 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-token\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.111345 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-credential-socket\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.111753 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.111682 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istiod-ca-cert\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.112243 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.111810 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-credential-socket\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.112243 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.111916 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-socket\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.113533 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.113512 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-envoy\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.113878 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.113858 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-podinfo\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.119657 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.119633 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59548\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-kube-api-access-59548\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.119657 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.119648 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-token\") pod \"e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.224347 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.224234 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:07.358187 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.356807 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p"] Jul 22 16:55:07.361444 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.361407 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:55:07.361544 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.361485 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:55:07.361544 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:07.361521 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:55:08.195799 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:08.195132 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" event={"ID":"a53219a9-1711-46c1-aaba-fa3b6fd8a19f","Type":"ContainerStarted","Data":"d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32"} Jul 22 16:55:08.195799 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:08.195225 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" event={"ID":"a53219a9-1711-46c1-aaba-fa3b6fd8a19f","Type":"ContainerStarted","Data":"d63fc8090d7b392cc2f2790f3f38b128f81ca647e3ddf4d7c37ca1df17b730fa"} Jul 22 16:55:08.214822 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:08.214780 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" podStartSLOduration=2.214767599 podStartE2EDuration="2.214767599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:55:08.213735949 +0000 UTC m=+1950.919178493" watchObservedRunningTime="2026-07-22 16:55:08.214767599 +0000 UTC m=+1950.920210119" Jul 22 16:55:08.224728 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:08.224701 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:08.229469 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:08.229448 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:09.200493 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:09.200465 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:09.201486 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:09.201461 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:10.142148 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:10.142113 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:10.234195 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:10.234161 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:10.308973 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:10.308943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:10.446718 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:10.446609 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:10.963430 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:10.963397 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:11.069164 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:11.069124 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:14.661370 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:14.661331 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:16.896998 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:16.896964 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:55:24.747350 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:24.746902 2555 patch_prober.go:28] interesting pod/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:56394->10.133.0.33:15021: read: connection reset by peer" start-of-body= Jul 22 16:55:24.747350 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:24.746985 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:56394->10.133.0.33:15021: read: connection reset by peer" Jul 22 16:55:25.269706 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:25.269670 2555 generic.go:356] "Generic (PLEG): container finished" podID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerID="d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32" exitCode=0 Jul 22 16:55:25.269943 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:25.269744 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" event={"ID":"a53219a9-1711-46c1-aaba-fa3b6fd8a19f","Type":"ContainerDied","Data":"d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32"} Jul 22 16:55:25.270292 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:25.270274 2555 scope.go:117] "RemoveContainer" containerID="d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32" Jul 22 16:55:25.271459 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:25.271404 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:55:25.271532 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:25.271517 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:55:25.271578 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:25.271563 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:55:26.276021 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:26.275987 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" event={"ID":"a53219a9-1711-46c1-aaba-fa3b6fd8a19f","Type":"ContainerStarted","Data":"58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e"} Jul 22 16:55:27.226040 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:27.225587 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:27.226040 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:27.225723 2555 patch_prober.go:28] interesting pod/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 22 16:55:27.226040 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:27.225777 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 22 16:55:27.332567 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:27.332528 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:27.684168 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:27.684129 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:55:28.225699 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:28.225653 2555 patch_prober.go:28] interesting pod/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 22 16:55:28.225931 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:28.225730 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 22 16:55:29.225335 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:29.225293 2555 patch_prober.go:28] interesting pod/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 22 16:55:29.225980 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:29.225946 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 22 16:55:30.228133 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:30.228103 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:30.228570 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:30.228551 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:30.229390 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:30.229373 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:55:30.299263 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:30.299215 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:55:49.792745 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:49.792710 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:55:53.277075 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.277042 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt"] Jul 22 16:55:53.277463 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.277367 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1518db-04e1-4af2-a59a-a8a8e7de5f5c" containerName="istio-proxy" Jul 22 16:55:53.280311 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.280295 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" Jul 22 16:55:53.282726 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.282701 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 16:55:53.282865 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.282709 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sh5d4\"" Jul 22 16:55:53.287164 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.287122 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt"] Jul 22 16:55:53.298727 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.298702 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt\" (UID: \"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" Jul 22 16:55:53.399796 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.399750 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt\" (UID: \"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" Jul 22 16:55:53.400403 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.400382 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt\" (UID: \"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" Jul 22 16:55:53.592945 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.592910 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" Jul 22 16:55:53.719560 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:53.719531 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt"] Jul 22 16:55:53.722813 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:55:53.722786 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff470bcb_1c11_4ef1_a4dd_c89e149dd2dd.slice/crio-109a6f2c137223d8665d50229ac2277751e46f220799619b1d91017a1431f686 WatchSource:0}: Error finding container 109a6f2c137223d8665d50229ac2277751e46f220799619b1d91017a1431f686: Status 404 returned error can't find the container with id 109a6f2c137223d8665d50229ac2277751e46f220799619b1d91017a1431f686 Jul 22 16:55:54.397827 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:54.397790 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" event={"ID":"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd","Type":"ContainerStarted","Data":"803aa502e47150fbe7d27472c3a9174aac89d9ee371461623ec9737d83b76f61"} Jul 22 16:55:54.398312 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:54.397841 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" event={"ID":"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd","Type":"ContainerStarted","Data":"109a6f2c137223d8665d50229ac2277751e46f220799619b1d91017a1431f686"} Jul 22 16:55:54.412667 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:54.412612 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" podStartSLOduration=1.412593285 podStartE2EDuration="1.412593285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:55:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:55:54.411050018 +0000 UTC m=+1997.116492559" watchObservedRunningTime="2026-07-22 16:55:54.412593285 +0000 UTC m=+1997.118035828" Jul 22 16:55:55.403337 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:55.403301 2555 generic.go:356] "Generic (PLEG): container finished" podID="ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd" containerID="803aa502e47150fbe7d27472c3a9174aac89d9ee371461623ec9737d83b76f61" exitCode=0 Jul 22 16:55:55.403747 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:55.403394 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" event={"ID":"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd","Type":"ContainerDied","Data":"803aa502e47150fbe7d27472c3a9174aac89d9ee371461623ec9737d83b76f61"} Jul 22 16:55:56.532279 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:56.532257 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" Jul 22 16:55:56.625353 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:56.625316 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd-maas-api-service-ca\") pod \"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd\" (UID: \"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd\") " Jul 22 16:55:56.625675 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:56.625649 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd" (UID: "ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:55:56.726721 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:56.726644 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd-maas-api-service-ca\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:55:57.413622 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:57.413579 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" event={"ID":"ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd","Type":"ContainerDied","Data":"109a6f2c137223d8665d50229ac2277751e46f220799619b1d91017a1431f686"} Jul 22 16:55:57.413831 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:57.413710 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="109a6f2c137223d8665d50229ac2277751e46f220799619b1d91017a1431f686" Jul 22 16:55:57.413831 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:55:57.413778 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt" Jul 22 16:56:07.298889 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:07.298844 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:56:07.657502 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:07.657415 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:08.725589 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:08.725552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:13.350172 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:13.350136 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:13.358507 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:13.358482 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt"] Jul 22 16:56:13.360336 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:13.360314 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b9fae547-s5tkt"] Jul 22 16:56:13.743820 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:13.743743 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd" path="/var/lib/kubelet/pods/ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd/volumes" Jul 22 16:56:15.688765 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:15.688732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:18.493540 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:18.493498 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:56:20.774705 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:20.774665 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:37.816909 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:37.816877 2555 scope.go:117] "RemoveContainer" containerID="ded0259e579a4e06a824190be28a6330776f92410dbffc1f000d331fd9a712e3" Jul 22 16:56:51.882535 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:51.882501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:51.983390 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:51.983347 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:57.001162 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:57.001128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:56:57.845384 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:56:57.845346 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:03.182304 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:03.182214 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:03.282702 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:03.282659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:03.461709 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:03.461634 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p"] Jul 22 16:57:03.461882 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:03.461857 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" containerID="cri-o://58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e" gracePeriod=30 Jul 22 16:57:03.485532 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:03.485503 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:03.748133 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:03.748029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:04.907063 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:04.907027 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:05.003222 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:05.003194 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:05.105758 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:05.105730 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:05.930687 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:05.930651 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:06.035068 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:06.035013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:06.133355 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:06.133315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:07.648255 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:07.648210 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:07.794391 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:07.794357 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:57:08.704495 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.704462 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:57:08.708974 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.708948 2555 generic.go:356] "Generic (PLEG): container finished" podID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerID="58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e" exitCode=0 Jul 22 16:57:08.709103 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.708993 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" Jul 22 16:57:08.709103 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.709034 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" event={"ID":"a53219a9-1711-46c1-aaba-fa3b6fd8a19f","Type":"ContainerDied","Data":"58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e"} Jul 22 16:57:08.709179 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.709109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p" event={"ID":"a53219a9-1711-46c1-aaba-fa3b6fd8a19f","Type":"ContainerDied","Data":"d63fc8090d7b392cc2f2790f3f38b128f81ca647e3ddf4d7c37ca1df17b730fa"} Jul 22 16:57:08.709179 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.709138 2555 scope.go:117] "RemoveContainer" containerID="58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e" Jul 22 16:57:08.716822 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.716806 2555 scope.go:117] "RemoveContainer" containerID="d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32" Jul 22 16:57:08.723995 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.723977 2555 scope.go:117] "RemoveContainer" containerID="58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e" Jul 22 16:57:08.724348 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:57:08.724307 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e\": container with ID starting with 58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e not found: ID does not exist" containerID="58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e" Jul 22 16:57:08.724459 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.724343 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e"} err="failed to get container status \"58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e\": rpc error: code = NotFound desc = could not find container \"58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e\": container with ID starting with 58373db931741b9e1f6108717f88c702f58435bf97348420b337ec070cbf1b2e not found: ID does not exist" Jul 22 16:57:08.724459 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.724369 2555 scope.go:117] "RemoveContainer" containerID="d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32" Jul 22 16:57:08.724857 ip-10-0-138-200 kubenswrapper[2555]: E0722 16:57:08.724829 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32\": container with ID starting with d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32 not found: ID does not exist" containerID="d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32" Jul 22 16:57:08.724971 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.724868 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32"} err="failed to get container status \"d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32\": rpc error: code = NotFound desc = could not find container \"d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32\": container with ID starting with d71cfef2a97adb6fc62f95551c6f031f0bddadd406770b23dd03ae83c9651e32 not found: ID does not exist" Jul 22 16:57:08.731922 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.731902 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-certs\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.731934 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-envoy\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.731951 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-credential-socket\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732010 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.731985 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-data\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732022 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-podinfo\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732045 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-token\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732110 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-socket\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732164 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59548\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-kube-api-access-59548\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732206 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732222 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732213 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istiod-ca-cert\") pod \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\" (UID: \"a53219a9-1711-46c1-aaba-fa3b6fd8a19f\") " Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732352 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:57:08.732528 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732516 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:57:08.733027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732532 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.733027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732551 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.733027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732566 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.733027 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.732938 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:57:08.734715 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.734645 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:57:08.734800 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.734762 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-kube-api-access-59548" (OuterVolumeSpecName: "kube-api-access-59548") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "kube-api-access-59548". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:57:08.734800 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.734757 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 16:57:08.735303 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.735273 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a53219a9-1711-46c1-aaba-fa3b6fd8a19f" (UID: "a53219a9-1711-46c1-aaba-fa3b6fd8a19f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:57:08.833231 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.833194 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.833231 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.833225 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.833231 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.833236 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.833476 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.833244 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.833476 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.833255 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59548\" (UniqueName: \"kubernetes.io/projected/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-kube-api-access-59548\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:08.833476 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:08.833265 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a53219a9-1711-46c1-aaba-fa3b6fd8a19f-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 16:57:09.030651 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:09.030626 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p"] Jul 22 16:57:09.034527 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:09.034505 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b9fae547-openshift-default-7f44c56df9-8v84p"] Jul 22 16:57:09.744038 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:09.744010 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" path="/var/lib/kubelet/pods/a53219a9-1711-46c1-aaba-fa3b6fd8a19f/volumes" Jul 22 16:57:13.557004 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:13.556964 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:13.619533 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:13.619498 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:14.258610 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:14.258567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:17.225550 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:17.225513 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:20.038562 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:20.038522 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:20.103264 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:20.103225 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:20.188525 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:20.188495 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:20.214435 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:20.214401 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:57:20.288802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:20.288715 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:30.823288 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:30.823245 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:31.204745 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:31.204659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:37.834417 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:37.834386 2555 scope.go:117] "RemoveContainer" containerID="e2ff47993838f271f2e5d41d1094203d6625aa92f1822bc1b6546d3a96986825" Jul 22 16:57:37.843443 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:37.843423 2555 scope.go:117] "RemoveContainer" containerID="d65daec75ceafa5c265fb8c9269844632102e6614d1c4e59ff8fa7b0bf743ff2" Jul 22 16:57:52.258892 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:52.258852 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:58.576148 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:58.576112 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:58.668584 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:58.668549 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:58.972722 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:58.972621 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:59.294097 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294055 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc"] Jul 22 16:57:59.294472 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294456 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd" containerName="revoke-keys" Jul 22 16:57:59.294524 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294475 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd" containerName="revoke-keys" Jul 22 16:57:59.294524 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294501 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" Jul 22 16:57:59.294524 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294510 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" Jul 22 16:57:59.294619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294523 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" Jul 22 16:57:59.294619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294532 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" Jul 22 16:57:59.294619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294597 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" Jul 22 16:57:59.294619 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.294607 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff470bcb-1c11-4ef1-a4dd-c89e149dd2dd" containerName="revoke-keys" Jul 22 16:57:59.297724 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.297706 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.300844 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.300823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b4259e-openshift-default-dockercfg-r8d5v\"" Jul 22 16:57:59.309688 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.309666 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc"] Jul 22 16:57:59.320337 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.320305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:59.359892 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.359855 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-data\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.359984 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.359898 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-credential-socket\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.359984 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.359929 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hxsv\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-kube-api-access-2hxsv\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.360057 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.360043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9554f3a7-b396-499a-8521-8de5178c7f28-istiod-ca-cert\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.360112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.360080 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-certs\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.360166 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.360147 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-socket\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.360202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.360175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-istio-token\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.360236 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.360213 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9554f3a7-b396-499a-8521-8de5178c7f28-istio-podinfo\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.360269 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.360248 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-envoy\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.456415 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.456374 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:57:59.461497 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461475 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9554f3a7-b396-499a-8521-8de5178c7f28-istiod-ca-cert\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461572 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461508 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-certs\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461572 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461532 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-socket\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-istio-token\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461640 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9554f3a7-b396-499a-8521-8de5178c7f28-istio-podinfo\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461724 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461638 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-envoy\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461724 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-data\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461724 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461702 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-credential-socket\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461873 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461731 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hxsv\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-kube-api-access-2hxsv\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.461931 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.461903 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-socket\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.462029 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.462006 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-certs\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.462175 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.462153 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-data\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.462227 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.462199 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-credential-socket\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.462339 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.462321 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9554f3a7-b396-499a-8521-8de5178c7f28-istiod-ca-cert\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.464145 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.464127 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-envoy\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.464212 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.464195 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9554f3a7-b396-499a-8521-8de5178c7f28-istio-podinfo\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.469611 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.469580 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-istio-token\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.469720 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.469648 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hxsv\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-kube-api-access-2hxsv\") pod \"e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.608939 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.608826 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:57:59.744723 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.744694 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc"] Jul 22 16:57:59.745667 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:57:59.745635 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9554f3a7_b396_499a_8521_8de5178c7f28.slice/crio-3e531b8fc3979d276fe8472bb748bc345cd5c4166180a7d898d9365970aa0ad1 WatchSource:0}: Error finding container 3e531b8fc3979d276fe8472bb748bc345cd5c4166180a7d898d9365970aa0ad1: Status 404 returned error can't find the container with id 3e531b8fc3979d276fe8472bb748bc345cd5c4166180a7d898d9365970aa0ad1 Jul 22 16:57:59.747685 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.747649 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:57:59.747743 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.747732 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:57:59.747788 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.747774 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:57:59.924820 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.924735 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" event={"ID":"9554f3a7-b396-499a-8521-8de5178c7f28","Type":"ContainerStarted","Data":"35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59"} Jul 22 16:57:59.924820 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.924812 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" event={"ID":"9554f3a7-b396-499a-8521-8de5178c7f28","Type":"ContainerStarted","Data":"3e531b8fc3979d276fe8472bb748bc345cd5c4166180a7d898d9365970aa0ad1"} Jul 22 16:57:59.943174 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:57:59.943127 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podStartSLOduration=0.943110121 podStartE2EDuration="943.110121ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:57:59.94268386 +0000 UTC m=+2122.648126415" watchObservedRunningTime="2026-07-22 16:57:59.943110121 +0000 UTC m=+2122.648552663" Jul 22 16:58:00.609735 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:00.609692 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:00.614450 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:00.614422 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:00.928845 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:00.928744 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:00.929620 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:00.929603 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:01.954527 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:01.954487 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:02.055184 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:02.055142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:02.177647 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:02.177612 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:02.366886 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:02.366849 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:02.634199 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:02.634121 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:02.738286 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:02.738245 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:06.525144 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:06.525109 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:06.881291 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:06.881248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:07.192984 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.186400 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2"] Jul 22 16:58:07.192984 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.187308 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a53219a9-1711-46c1-aaba-fa3b6fd8a19f" containerName="istio-proxy" Jul 22 16:58:07.192984 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.191582 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.195386 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.195361 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6608f1-openshift-default-dockercfg-gjgl9\"" Jul 22 16:58:07.209114 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.209074 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2"] Jul 22 16:58:07.222809 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.222779 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:07.327924 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.327888 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb2r2\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-kube-api-access-kb2r2\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.327942 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-credential-socket\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.327960 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-token\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.328031 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-certs\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.328061 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-data\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.328109 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-socket\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328128 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.328124 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-envoy\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.328138 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-podinfo\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.328325 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.328153 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c606dea-1c2a-41d5-b971-cd5e4767e690-istiod-ca-cert\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.377437 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.377389 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:07.428624 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428593 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-certs\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.428802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428641 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-data\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.428802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428680 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-socket\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.428802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428702 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-envoy\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.428802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428717 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-podinfo\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.428802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c606dea-1c2a-41d5-b971-cd5e4767e690-istiod-ca-cert\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.428802 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428788 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb2r2\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-kube-api-access-kb2r2\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.429141 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428871 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-credential-socket\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.429141 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.428909 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-token\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.429141 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.429059 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-certs\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.429141 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.429111 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-socket\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.429345 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.429159 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-data\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.429396 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.429377 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-credential-socket\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.429553 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.429535 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c606dea-1c2a-41d5-b971-cd5e4767e690-istiod-ca-cert\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.431759 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.431726 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-envoy\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.431912 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.431893 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-podinfo\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.437281 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.437253 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-token\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.437546 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.437519 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb2r2\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-kube-api-access-kb2r2\") pod \"e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.503623 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.503538 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:07.640991 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.640969 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2"] Jul 22 16:58:07.643417 ip-10-0-138-200 kubenswrapper[2555]: W0722 16:58:07.643391 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c606dea_1c2a_41d5_b971_cd5e4767e690.slice/crio-b608bbd3b1f4ee7dac5aca97ca7c487f50b23b4f2080d10494437374912f81b3 WatchSource:0}: Error finding container b608bbd3b1f4ee7dac5aca97ca7c487f50b23b4f2080d10494437374912f81b3: Status 404 returned error can't find the container with id b608bbd3b1f4ee7dac5aca97ca7c487f50b23b4f2080d10494437374912f81b3 Jul 22 16:58:07.645429 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.645393 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:07.645537 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.645475 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:07.645537 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.645518 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:07.956299 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.956261 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" event={"ID":"7c606dea-1c2a-41d5-b971-cd5e4767e690","Type":"ContainerStarted","Data":"b649a4c040a3956a5ba72369939e627d60422cb9304afa3b7c19bfc02b1cea2b"} Jul 22 16:58:07.956474 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.956315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" event={"ID":"7c606dea-1c2a-41d5-b971-cd5e4767e690","Type":"ContainerStarted","Data":"b608bbd3b1f4ee7dac5aca97ca7c487f50b23b4f2080d10494437374912f81b3"} Jul 22 16:58:07.981750 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:07.981702 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podStartSLOduration=0.981685956 podStartE2EDuration="981.685956ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:58:07.979891825 +0000 UTC m=+2130.685334368" watchObservedRunningTime="2026-07-22 16:58:07.981685956 +0000 UTC m=+2130.687128502" Jul 22 16:58:08.504533 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:08.504493 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:08.509378 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:08.509351 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:08.960792 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:08.960757 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:08.961621 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:08.961601 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:09.595907 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:09.595872 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:58:09.967127 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:09.967032 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:10.061470 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:10.061426 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:10.161698 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:10.161652 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:10.462445 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:10.462408 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:10.564951 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:10.564911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:10.662556 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:10.662513 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:14.496804 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:14.496767 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:16.970267 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:16.970213 2555 patch_prober.go:28] interesting pod/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 16:58:16.970746 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:16.970289 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 16:58:17.999609 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:17.999518 2555 generic.go:356] "Generic (PLEG): container finished" podID="9554f3a7-b396-499a-8521-8de5178c7f28" containerID="35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59" exitCode=0 Jul 22 16:58:17.999609 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:17.999589 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" event={"ID":"9554f3a7-b396-499a-8521-8de5178c7f28","Type":"ContainerDied","Data":"35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59"} Jul 22 16:58:18.000022 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:17.999952 2555 scope.go:117] "RemoveContainer" containerID="35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59" Jul 22 16:58:18.000941 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:18.000917 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:18.000992 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:18.000980 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:18.001032 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:18.001021 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:19.006221 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:19.006174 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" event={"ID":"9554f3a7-b396-499a-8521-8de5178c7f28","Type":"ContainerStarted","Data":"8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b"} Jul 22 16:58:19.609595 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:19.609549 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:19.609822 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:19.609685 2555 patch_prober.go:28] interesting pod/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 22 16:58:19.609822 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:19.609739 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 22 16:58:20.609693 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:20.609653 2555 patch_prober.go:28] interesting pod/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 22 16:58:20.610197 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:20.609725 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 22 16:58:21.363214 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:21.363175 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:58:21.610055 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:21.610025 2555 patch_prober.go:28] interesting pod/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 22 16:58:21.610624 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:21.610570 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 22 16:58:22.609614 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:22.609573 2555 patch_prober.go:28] interesting pod/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 22 16:58:22.609961 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:22.609930 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 22 16:58:23.323945 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:23.323899 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:23.426439 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:23.426403 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:23.528000 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:23.527954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:23.614330 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:23.614232 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:23.614644 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:23.614611 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:23.615713 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:23.615692 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 16:58:24.386167 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:24.385253 2555 patch_prober.go:28] interesting pod/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:56042->10.133.0.36:15021: read: connection reset by peer" start-of-body= Jul 22 16:58:24.386167 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:24.385328 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:56042->10.133.0.36:15021: read: connection reset by peer" Jul 22 16:58:25.040597 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:25.040565 2555 generic.go:356] "Generic (PLEG): container finished" podID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerID="b649a4c040a3956a5ba72369939e627d60422cb9304afa3b7c19bfc02b1cea2b" exitCode=0 Jul 22 16:58:25.040774 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:25.040634 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" event={"ID":"7c606dea-1c2a-41d5-b971-cd5e4767e690","Type":"ContainerDied","Data":"b649a4c040a3956a5ba72369939e627d60422cb9304afa3b7c19bfc02b1cea2b"} Jul 22 16:58:25.041185 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:25.041164 2555 scope.go:117] "RemoveContainer" containerID="b649a4c040a3956a5ba72369939e627d60422cb9304afa3b7c19bfc02b1cea2b" Jul 22 16:58:25.042165 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:25.042132 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:25.042287 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:25.042199 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:25.042287 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:25.042230 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 16:58:26.046686 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:26.046654 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" event={"ID":"7c606dea-1c2a-41d5-b971-cd5e4767e690","Type":"ContainerStarted","Data":"35a4db2466aabba1fb5b761819ad709fbe04da4d1fc7fa1387a2c9d3923bcf01"} Jul 22 16:58:26.504518 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:26.504409 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:26.504708 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:26.504554 2555 patch_prober.go:28] interesting pod/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 22 16:58:26.504708 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:26.504614 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 22 16:58:27.504327 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:27.504284 2555 patch_prober.go:28] interesting pod/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 22 16:58:27.504950 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:27.504915 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 22 16:58:28.504489 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:28.504381 2555 patch_prober.go:28] interesting pod/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 22 16:58:28.505112 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:28.505053 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 22 16:58:29.504985 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:29.504944 2555 patch_prober.go:28] interesting pod/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 22 16:58:29.505641 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:29.505607 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 22 16:58:30.507803 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:30.507770 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:30.508223 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:30.508064 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:30.508826 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:30.508807 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 16:58:31.264242 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:31.264215 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:31.369516 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:31.369476 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:58:33.298575 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:33.298539 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:58:37.865735 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:58:37.865694 2555 scope.go:117] "RemoveContainer" containerID="b9436aa4fcef8c46cbfdbbe86e21faa3012566ce0f96586c16dbd7064bddfb56" Jul 22 16:59:14.823202 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:59:14.823165 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 16:59:19.189349 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:59:19.189309 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 16:59:33.404337 ip-10-0-138-200 kubenswrapper[2555]: I0722 16:59:33.404294 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:00:15.796154 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:15.796120 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:16.754661 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:16.754619 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:17.854465 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:17.854417 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:20.894909 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:20.894873 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:00:27.378215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:27.378165 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:31.944188 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:31.944148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:32.352654 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:32.352617 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:33.194627 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:33.194592 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:00:35.264960 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:35.264915 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:37.894460 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:37.894428 2555 scope.go:117] "RemoveContainer" containerID="81675024e8e9d9410444c839910b78329a936d026291cbc0f7afd39e1671f03f" Jul 22 17:00:38.023511 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.023477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:38.122800 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.122759 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:38.266069 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.265977 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc"] Jul 22 17:00:38.266349 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.266322 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" containerID="cri-o://8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b" gracePeriod=30 Jul 22 17:00:38.379495 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.379460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:38.579132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.579082 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:38.616811 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.616770 2555 patch_prober.go:28] interesting pod/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 17:00:38.616966 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:38.616839 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 17:00:39.322660 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:39.322622 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:39.423596 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:39.423554 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:43.512081 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.512057 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 17:00:43.546200 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546174 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-data\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546391 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546214 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-certs\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546391 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546252 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hxsv\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-kube-api-access-2hxsv\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546391 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546272 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-istio-token\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546391 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546307 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-credential-socket\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546391 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546327 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9554f3a7-b396-499a-8521-8de5178c7f28-istiod-ca-cert\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546675 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546463 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-socket\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546675 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546523 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9554f3a7-b396-499a-8521-8de5178c7f28-istio-podinfo\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546675 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546562 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-envoy\") pod \"9554f3a7-b396-499a-8521-8de5178c7f28\" (UID: \"9554f3a7-b396-499a-8521-8de5178c7f28\") " Jul 22 17:00:43.546675 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546648 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:00:43.546878 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546676 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:00:43.546878 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546795 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:00:43.546878 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546872 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.547025 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546889 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.547025 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.546904 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.547155 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.547045 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-data" (OuterVolumeSpecName: "istio-data") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:00:43.547323 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.547286 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9554f3a7-b396-499a-8521-8de5178c7f28-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 17:00:43.548985 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.548962 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:00:43.549214 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.549194 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-istio-token" (OuterVolumeSpecName: "istio-token") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:00:43.549288 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.549243 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9554f3a7-b396-499a-8521-8de5178c7f28-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 17:00:43.549288 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.549258 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-kube-api-access-2hxsv" (OuterVolumeSpecName: "kube-api-access-2hxsv") pod "9554f3a7-b396-499a-8521-8de5178c7f28" (UID: "9554f3a7-b396-499a-8521-8de5178c7f28"). InnerVolumeSpecName "kube-api-access-2hxsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:00:43.604810 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.604774 2555 generic.go:356] "Generic (PLEG): container finished" podID="9554f3a7-b396-499a-8521-8de5178c7f28" containerID="8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b" exitCode=0 Jul 22 17:00:43.604991 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.604823 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" Jul 22 17:00:43.604991 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.604825 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" event={"ID":"9554f3a7-b396-499a-8521-8de5178c7f28","Type":"ContainerDied","Data":"8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b"} Jul 22 17:00:43.604991 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.604882 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc" event={"ID":"9554f3a7-b396-499a-8521-8de5178c7f28","Type":"ContainerDied","Data":"3e531b8fc3979d276fe8472bb748bc345cd5c4166180a7d898d9365970aa0ad1"} Jul 22 17:00:43.604991 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.604910 2555 scope.go:117] "RemoveContainer" containerID="8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b" Jul 22 17:00:43.614263 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.614245 2555 scope.go:117] "RemoveContainer" containerID="35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59" Jul 22 17:00:43.621672 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.621654 2555 scope.go:117] "RemoveContainer" containerID="8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b" Jul 22 17:00:43.621933 ip-10-0-138-200 kubenswrapper[2555]: E0722 17:00:43.621913 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b\": container with ID starting with 8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b not found: ID does not exist" containerID="8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b" Jul 22 17:00:43.622003 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.621942 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b"} err="failed to get container status \"8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b\": rpc error: code = NotFound desc = could not find container \"8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b\": container with ID starting with 8bb2cdf335016c707c50e361f202be356ec16902d9297a277214f71e50744f8b not found: ID does not exist" Jul 22 17:00:43.622003 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.621957 2555 scope.go:117] "RemoveContainer" containerID="35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59" Jul 22 17:00:43.622224 ip-10-0-138-200 kubenswrapper[2555]: E0722 17:00:43.622205 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59\": container with ID starting with 35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59 not found: ID does not exist" containerID="35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59" Jul 22 17:00:43.622269 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.622232 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59"} err="failed to get container status \"35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59\": rpc error: code = NotFound desc = could not find container \"35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59\": container with ID starting with 35d22d53eca8d733f85f6a6e19a70c77230a7b33fb2718c784128bda0ce89e59 not found: ID does not exist" Jul 22 17:00:43.632222 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.632197 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc"] Jul 22 17:00:43.637817 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.637795 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b4259e-openshift-default-7645cb4bbc-xr9wc"] Jul 22 17:00:43.647984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.647959 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9554f3a7-b396-499a-8521-8de5178c7f28-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.648113 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.647985 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.648113 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.647999 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9554f3a7-b396-499a-8521-8de5178c7f28-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.648113 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.648011 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hxsv\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-kube-api-access-2hxsv\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.648113 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.648023 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9554f3a7-b396-499a-8521-8de5178c7f28-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.648113 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.648034 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9554f3a7-b396-499a-8521-8de5178c7f28-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:00:43.743502 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:43.743470 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" path="/var/lib/kubelet/pods/9554f3a7-b396-499a-8521-8de5178c7f28/volumes" Jul 22 17:00:49.207791 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:49.207752 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:52.091233 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:52.091197 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:00:53.743673 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:53.743636 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:00:54.293799 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:00:54.293758 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:00.185157 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.185113 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:00.368739 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.368686 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:00.468807 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.468714 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2"] Jul 22 17:01:00.469045 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.469002 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" containerID="cri-o://35a4db2466aabba1fb5b761819ad709fbe04da4d1fc7fa1387a2c9d3923bcf01" gracePeriod=30 Jul 22 17:01:00.499062 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.499025 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:00.510397 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.510363 2555 patch_prober.go:28] interesting pod/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 17:01:00.510522 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.510422 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 17:01:00.722709 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:00.722615 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:01.089281 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.089244 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:01.383640 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.383566 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq"] Jul 22 17:01:01.384013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.383866 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" Jul 22 17:01:01.384013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.383879 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" Jul 22 17:01:01.384013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.383890 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" Jul 22 17:01:01.384013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.383896 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" Jul 22 17:01:01.384013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.383953 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" Jul 22 17:01:01.384013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.383960 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9554f3a7-b396-499a-8521-8de5178c7f28" containerName="istio-proxy" Jul 22 17:01:01.388073 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.388054 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.391111 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.391067 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e1e63b-openshift-default-dockercfg-m9l9s\"" Jul 22 17:01:01.402705 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.402678 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq"] Jul 22 17:01:01.419356 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.419320 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:01.488238 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488203 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-data\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488238 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488242 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f555c114-1a91-4f85-88d6-45c09502f094-istiod-ca-cert\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488265 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-envoy\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-certs\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488371 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-credential-socket\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488390 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f555c114-1a91-4f85-88d6-45c09502f094-istio-podinfo\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488407 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-istio-token\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488439 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-socket\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.488658 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.488468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5fnh\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-kube-api-access-n5fnh\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.513168 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.513126 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:01.589294 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-socket\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589509 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589358 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5fnh\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-kube-api-access-n5fnh\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589509 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589404 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-data\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589509 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589433 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f555c114-1a91-4f85-88d6-45c09502f094-istiod-ca-cert\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589509 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589456 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-envoy\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589758 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589515 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-certs\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589758 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589550 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-credential-socket\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589758 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589575 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f555c114-1a91-4f85-88d6-45c09502f094-istio-podinfo\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589758 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589602 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-istio-token\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589758 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589736 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-socket\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.589758 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.589750 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-data\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.590029 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.590006 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-credential-socket\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.590066 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.590032 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-certs\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.590202 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.590181 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f555c114-1a91-4f85-88d6-45c09502f094-istiod-ca-cert\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.591797 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.591777 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-envoy\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.592202 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.592184 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f555c114-1a91-4f85-88d6-45c09502f094-istio-podinfo\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.598396 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.598373 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-istio-token\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.598529 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.598505 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5fnh\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-kube-api-access-n5fnh\") pod \"e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.697462 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.697367 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:01.830724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.830683 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq"] Jul 22 17:01:01.832865 ip-10-0-138-200 kubenswrapper[2555]: W0722 17:01:01.832836 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf555c114_1a91_4f85_88d6_45c09502f094.slice/crio-60047f8c0d6726ce4109a73b5d7e7f56ed3ab96154280cfc326715ed9adf9d11 WatchSource:0}: Error finding container 60047f8c0d6726ce4109a73b5d7e7f56ed3ab96154280cfc326715ed9adf9d11: Status 404 returned error can't find the container with id 60047f8c0d6726ce4109a73b5d7e7f56ed3ab96154280cfc326715ed9adf9d11 Jul 22 17:01:01.834845 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.834816 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:01.834931 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.834883 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:01.834931 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:01.834912 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:02.685340 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:02.685303 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" event={"ID":"f555c114-1a91-4f85-88d6-45c09502f094","Type":"ContainerStarted","Data":"b5095a258af32888e0d695900b5e6ceb5da043eef7cfb6a1770a3b628d6b8618"} Jul 22 17:01:02.685812 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:02.685367 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" event={"ID":"f555c114-1a91-4f85-88d6-45c09502f094","Type":"ContainerStarted","Data":"60047f8c0d6726ce4109a73b5d7e7f56ed3ab96154280cfc326715ed9adf9d11"} Jul 22 17:01:02.698078 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:02.698055 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:02.703055 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:02.703035 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:02.705127 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:02.705068 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" podStartSLOduration=1.705053934 podStartE2EDuration="1.705053934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:01:02.703972144 +0000 UTC m=+2305.409414699" watchObservedRunningTime="2026-07-22 17:01:02.705053934 +0000 UTC m=+2305.410496477" Jul 22 17:01:03.689713 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:03.689677 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:03.690582 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:03.690560 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:04.633190 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:04.633153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:04.716282 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:04.716240 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:04.793642 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:04.793608 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:04.926916 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:04.926815 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:05.205439 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.205339 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:05.334573 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.334532 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:05.700974 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.700940 2555 generic.go:356] "Generic (PLEG): container finished" podID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerID="35a4db2466aabba1fb5b761819ad709fbe04da4d1fc7fa1387a2c9d3923bcf01" exitCode=0 Jul 22 17:01:05.701171 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.701024 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" event={"ID":"7c606dea-1c2a-41d5-b971-cd5e4767e690","Type":"ContainerDied","Data":"35a4db2466aabba1fb5b761819ad709fbe04da4d1fc7fa1387a2c9d3923bcf01"} Jul 22 17:01:05.701171 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.701164 2555 scope.go:117] "RemoveContainer" containerID="b649a4c040a3956a5ba72369939e627d60422cb9304afa3b7c19bfc02b1cea2b" Jul 22 17:01:05.731304 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.731254 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 17:01:05.831834 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.831782 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-credential-socket\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.831834 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.831828 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-certs\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832109 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.831859 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-socket\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832109 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.831899 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-envoy\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832109 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.831915 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c606dea-1c2a-41d5-b971-cd5e4767e690-istiod-ca-cert\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832109 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.831933 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-podinfo\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832109 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.831983 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-data\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832109 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832011 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-token\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832109 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832040 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb2r2\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-kube-api-access-kb2r2\") pod \"7c606dea-1c2a-41d5-b971-cd5e4767e690\" (UID: \"7c606dea-1c2a-41d5-b971-cd5e4767e690\") " Jul 22 17:01:05.832462 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832108 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:05.832462 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832189 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:05.832462 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832381 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.832462 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832402 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.832462 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832378 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c606dea-1c2a-41d5-b971-cd5e4767e690-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 17:01:05.832690 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832542 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:05.832690 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.832668 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:05.834377 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.834351 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 17:01:05.834522 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.834503 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-kube-api-access-kb2r2" (OuterVolumeSpecName: "kube-api-access-kb2r2") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "kube-api-access-kb2r2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:01:05.834582 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.834523 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:01:05.834646 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.834633 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c606dea-1c2a-41d5-b971-cd5e4767e690" (UID: "7c606dea-1c2a-41d5-b971-cd5e4767e690"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:05.932926 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.932895 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.932926 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.932923 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.932926 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.932932 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c606dea-1c2a-41d5-b971-cd5e4767e690-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.933174 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.932940 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.933174 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.932948 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.933174 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.932958 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:05.933174 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:05.932965 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb2r2\" (UniqueName: \"kubernetes.io/projected/7c606dea-1c2a-41d5-b971-cd5e4767e690-kube-api-access-kb2r2\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:06.706769 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:06.706738 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" Jul 22 17:01:06.706769 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:06.706745 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2" event={"ID":"7c606dea-1c2a-41d5-b971-cd5e4767e690","Type":"ContainerDied","Data":"b608bbd3b1f4ee7dac5aca97ca7c487f50b23b4f2080d10494437374912f81b3"} Jul 22 17:01:06.707039 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:06.706800 2555 scope.go:117] "RemoveContainer" containerID="35a4db2466aabba1fb5b761819ad709fbe04da4d1fc7fa1387a2c9d3923bcf01" Jul 22 17:01:06.727704 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:06.727673 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2"] Jul 22 17:01:06.731223 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:06.731199 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6608f1-openshift-default-7485777db9-ffln2"] Jul 22 17:01:07.743520 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:07.743486 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" path="/var/lib/kubelet/pods/7c606dea-1c2a-41d5-b971-cd5e4767e690/volumes" Jul 22 17:01:09.157246 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.157215 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:09.635125 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.635062 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:09.933901 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.933816 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6"] Jul 22 17:01:09.934204 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.934185 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" Jul 22 17:01:09.934305 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.934205 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" Jul 22 17:01:09.934369 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.934306 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" Jul 22 17:01:09.934369 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.934318 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" Jul 22 17:01:09.938566 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.938542 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:09.941508 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.941489 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-828d6a-openshift-default-dockercfg-rwmnh\"" Jul 22 17:01:09.946218 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.946199 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6"] Jul 22 17:01:09.967422 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:09.967398 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:10.066862 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.066828 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-data\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.066862 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.066864 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-socket\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.067075 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.066891 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83972a08-f752-460f-a95a-febaaef4a2b7-istio-podinfo\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.067075 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.067004 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-istio-token\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.067075 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.067041 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83972a08-f752-460f-a95a-febaaef4a2b7-istiod-ca-cert\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.067075 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.067072 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-credential-socket\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.067250 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.067128 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-envoy\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.067250 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.067160 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-certs\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.067250 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.067194 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8d6t\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-kube-api-access-s8d6t\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.098553 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.098510 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:10.168402 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168363 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-istio-token\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168402 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168403 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83972a08-f752-460f-a95a-febaaef4a2b7-istiod-ca-cert\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168425 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-credential-socket\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-envoy\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168473 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-certs\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168498 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8d6t\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-kube-api-access-s8d6t\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168523 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-data\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168558 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-socket\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168590 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83972a08-f752-460f-a95a-febaaef4a2b7-istio-podinfo\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.168929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168916 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-credential-socket\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.169498 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.168968 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-certs\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.169498 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.169016 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-data\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.169498 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.169132 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-socket\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.169498 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.169191 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83972a08-f752-460f-a95a-febaaef4a2b7-istiod-ca-cert\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.171132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.171107 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-envoy\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.171311 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.171295 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83972a08-f752-460f-a95a-febaaef4a2b7-istio-podinfo\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.176356 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.176334 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8d6t\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-kube-api-access-s8d6t\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.176469 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.176449 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-istio-token\") pod \"e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.249946 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.249864 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:10.384317 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.384293 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6"] Jul 22 17:01:10.386585 ip-10-0-138-200 kubenswrapper[2555]: W0722 17:01:10.386552 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83972a08_f752_460f_a95a_febaaef4a2b7.slice/crio-56c1871d4d950973371a0a28dc89ecb407620d45f7217cad80a872696bf1d7c7 WatchSource:0}: Error finding container 56c1871d4d950973371a0a28dc89ecb407620d45f7217cad80a872696bf1d7c7: Status 404 returned error can't find the container with id 56c1871d4d950973371a0a28dc89ecb407620d45f7217cad80a872696bf1d7c7 Jul 22 17:01:10.388788 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.388757 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:10.388872 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.388823 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:10.388872 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.388851 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:10.728146 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.728113 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" event={"ID":"83972a08-f752-460f-a95a-febaaef4a2b7","Type":"ContainerStarted","Data":"6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713"} Jul 22 17:01:10.728301 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.728163 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" event={"ID":"83972a08-f752-460f-a95a-febaaef4a2b7","Type":"ContainerStarted","Data":"56c1871d4d950973371a0a28dc89ecb407620d45f7217cad80a872696bf1d7c7"} Jul 22 17:01:10.748492 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:10.748443 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" podStartSLOduration=1.748428427 podStartE2EDuration="1.748428427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:01:10.746822486 +0000 UTC m=+2313.452265022" watchObservedRunningTime="2026-07-22 17:01:10.748428427 +0000 UTC m=+2313.453870968" Jul 22 17:01:11.250107 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:11.250056 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:11.254694 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:11.254668 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:11.701590 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:11.701559 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:01:11.732699 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:11.732671 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:11.733472 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:11.733457 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:12.759159 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:12.759105 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:12.856824 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:12.856772 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:13.030605 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:13.030548 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:13.232079 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:13.232038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:13.634004 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:13.633956 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:13.738177 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:13.738138 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:17.224415 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:17.224371 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:19.444690 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.443961 2555 patch_prober.go:28] interesting pod/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:34464->10.133.0.37:15021: read: connection reset by peer" start-of-body= Jul 22 17:01:19.444690 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.444036 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:34464->10.133.0.37:15021: read: connection reset by peer" Jul 22 17:01:19.768845 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.768753 2555 generic.go:356] "Generic (PLEG): container finished" podID="f555c114-1a91-4f85-88d6-45c09502f094" containerID="b5095a258af32888e0d695900b5e6ceb5da043eef7cfb6a1770a3b628d6b8618" exitCode=0 Jul 22 17:01:19.768845 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.768825 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" event={"ID":"f555c114-1a91-4f85-88d6-45c09502f094","Type":"ContainerDied","Data":"b5095a258af32888e0d695900b5e6ceb5da043eef7cfb6a1770a3b628d6b8618"} Jul 22 17:01:19.769203 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.769189 2555 scope.go:117] "RemoveContainer" containerID="b5095a258af32888e0d695900b5e6ceb5da043eef7cfb6a1770a3b628d6b8618" Jul 22 17:01:19.770258 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.770231 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:19.770344 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.770324 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:19.770393 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:19.770378 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:20.774889 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:20.774848 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" event={"ID":"f555c114-1a91-4f85-88d6-45c09502f094","Type":"ContainerStarted","Data":"4ee554ffd5b642c67e41fbc8a1840fe92816bb9e8ad27c641b9319acc4fbfc5e"} Jul 22 17:01:21.698640 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:21.698602 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:21.698640 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:21.698643 2555 patch_prober.go:28] interesting pod/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 22 17:01:21.698895 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:21.698698 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 22 17:01:22.698408 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:22.698370 2555 patch_prober.go:28] interesting pod/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 22 17:01:22.698818 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:22.698427 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 22 17:01:23.403061 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:23.403026 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:01:23.698420 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:23.698337 2555 patch_prober.go:28] interesting pod/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 22 17:01:23.698420 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:23.698393 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 22 17:01:24.701311 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:24.701274 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:24.701778 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:24.701757 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:24.702555 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:24.702530 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:25.896134 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:25.896075 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:25.996891 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:25.996855 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:26.734647 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.734610 2555 patch_prober.go:28] interesting pod/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 22 17:01:26.734855 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.734676 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 22 17:01:26.802059 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.802025 2555 generic.go:356] "Generic (PLEG): container finished" podID="83972a08-f752-460f-a95a-febaaef4a2b7" containerID="6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713" exitCode=0 Jul 22 17:01:26.802239 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.802109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" event={"ID":"83972a08-f752-460f-a95a-febaaef4a2b7","Type":"ContainerDied","Data":"6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713"} Jul 22 17:01:26.802483 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.802469 2555 scope.go:117] "RemoveContainer" containerID="6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713" Jul 22 17:01:26.803412 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.803384 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:26.803481 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.803468 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:26.803538 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:26.803526 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:01:27.811114 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:27.811052 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" event={"ID":"83972a08-f752-460f-a95a-febaaef4a2b7","Type":"ContainerStarted","Data":"fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee"} Jul 22 17:01:28.250378 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:28.250290 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:28.250524 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:28.250484 2555 patch_prober.go:28] interesting pod/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 22 17:01:28.250565 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:28.250528 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 22 17:01:29.251051 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:29.251010 2555 patch_prober.go:28] interesting pod/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 22 17:01:29.251476 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:29.251071 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 22 17:01:30.250787 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:30.250747 2555 patch_prober.go:28] interesting pod/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 22 17:01:30.250983 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:30.250804 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 22 17:01:31.250463 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:31.250427 2555 patch_prober.go:28] interesting pod/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 22 17:01:31.250874 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:31.250485 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 22 17:01:32.254799 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:32.254765 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:32.255285 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:32.255029 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:32.255827 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:32.255805 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:01:35.195059 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:35.195021 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:35.296424 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:35.296390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:40.473693 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:40.473657 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:42.994577 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:42.994539 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:01:45.565569 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:45.565531 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:45.871345 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:45.871246 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:51.456741 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:51.456697 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:51.557189 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:51.557155 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:51.784079 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:51.784045 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq"] Jul 22 17:01:51.784312 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:51.784290 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" containerID="cri-o://4ee554ffd5b642c67e41fbc8a1840fe92816bb9e8ad27c641b9319acc4fbfc5e" gracePeriod=30 Jul 22 17:01:51.792445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:51.792419 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:51.976464 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:51.976420 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:53.322051 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:53.322016 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:53.422960 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:53.422918 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:54.111486 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:54.111448 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:54.210844 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:54.210809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:54.704682 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:54.704642 2555 patch_prober.go:28] interesting pod/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 17:01:54.705057 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:54.704709 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 17:01:54.901108 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:54.901051 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:55.004022 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:55.003919 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:55.725198 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:55.725163 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:55.825863 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:55.825827 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:56.499853 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:56.499809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:56.601274 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:56.601241 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:56.941275 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:56.941244 2555 generic.go:356] "Generic (PLEG): container finished" podID="f555c114-1a91-4f85-88d6-45c09502f094" containerID="4ee554ffd5b642c67e41fbc8a1840fe92816bb9e8ad27c641b9319acc4fbfc5e" exitCode=0 Jul 22 17:01:56.941672 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:56.941308 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" event={"ID":"f555c114-1a91-4f85-88d6-45c09502f094","Type":"ContainerDied","Data":"4ee554ffd5b642c67e41fbc8a1840fe92816bb9e8ad27c641b9319acc4fbfc5e"} Jul 22 17:01:56.941672 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:56.941368 2555 scope.go:117] "RemoveContainer" containerID="b5095a258af32888e0d695900b5e6ceb5da043eef7cfb6a1770a3b628d6b8618" Jul 22 17:01:57.028680 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.028655 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:57.070569 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070499 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-credential-socket\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.070569 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070560 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-istio-token\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.070759 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070596 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-envoy\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.070759 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070621 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f555c114-1a91-4f85-88d6-45c09502f094-istio-podinfo\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.070759 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070645 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5fnh\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-kube-api-access-n5fnh\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.070759 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070723 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-certs\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.070759 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070748 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-data\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.071022 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070781 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f555c114-1a91-4f85-88d6-45c09502f094-istiod-ca-cert\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.071022 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070777 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:57.071022 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.070837 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-socket\") pod \"f555c114-1a91-4f85-88d6-45c09502f094\" (UID: \"f555c114-1a91-4f85-88d6-45c09502f094\") " Jul 22 17:01:57.071022 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.071011 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:57.071260 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.071075 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.071260 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.071124 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.071505 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.071322 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f555c114-1a91-4f85-88d6-45c09502f094-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 17:01:57.071505 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.071335 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:57.072121 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.072067 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-data" (OuterVolumeSpecName: "istio-data") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:57.073056 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.073025 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f555c114-1a91-4f85-88d6-45c09502f094-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 17:01:57.073386 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.073354 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-istio-token" (OuterVolumeSpecName: "istio-token") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:01:57.073486 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.073408 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-kube-api-access-n5fnh" (OuterVolumeSpecName: "kube-api-access-n5fnh") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "kube-api-access-n5fnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:01:57.073486 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.073416 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f555c114-1a91-4f85-88d6-45c09502f094" (UID: "f555c114-1a91-4f85-88d6-45c09502f094"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:01:57.172208 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.172170 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.172208 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.172202 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.172208 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.172213 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.172420 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.172224 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f555c114-1a91-4f85-88d6-45c09502f094-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.172420 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.172235 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5fnh\" (UniqueName: \"kubernetes.io/projected/f555c114-1a91-4f85-88d6-45c09502f094-kube-api-access-n5fnh\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.172420 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.172248 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f555c114-1a91-4f85-88d6-45c09502f094-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.172420 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.172259 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f555c114-1a91-4f85-88d6-45c09502f094-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:01:57.438710 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.438617 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:57.540179 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.540139 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:57.947511 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.947479 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" Jul 22 17:01:57.947511 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.947501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq" event={"ID":"f555c114-1a91-4f85-88d6-45c09502f094","Type":"ContainerDied","Data":"60047f8c0d6726ce4109a73b5d7e7f56ed3ab96154280cfc326715ed9adf9d11"} Jul 22 17:01:57.947986 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.947554 2555 scope.go:117] "RemoveContainer" containerID="4ee554ffd5b642c67e41fbc8a1840fe92816bb9e8ad27c641b9319acc4fbfc5e" Jul 22 17:01:57.965870 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.965840 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq"] Jul 22 17:01:57.970507 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:57.970484 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e1e63b-openshift-default-9d5588bf9-mmlcq"] Jul 22 17:01:58.212410 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:58.212366 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:58.311689 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:58.311661 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:59.192740 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:59.192711 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:59.294445 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:59.294406 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:01:59.744078 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:59.744047 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f555c114-1a91-4f85-88d6-45c09502f094" path="/var/lib/kubelet/pods/f555c114-1a91-4f85-88d6-45c09502f094/volumes" Jul 22 17:01:59.925846 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:01:59.925816 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:00.027210 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:00.027123 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:00.784913 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:00.784883 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:00.889755 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:00.889720 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:01.531809 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:01.531775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:01.632506 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:01.632472 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:02.329077 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:02.329033 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:02.435471 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:02.435430 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:03.149953 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:03.149918 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:03.247846 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:03.247817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:08.410306 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:08.410257 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:12.200253 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:12.200219 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:12.602174 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:12.602135 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:13.495561 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:13.495527 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:02:18.437656 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:18.437625 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:18.537957 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:18.537923 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:18.672298 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:18.672266 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6"] Jul 22 17:02:18.672543 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:18.672501 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" containerID="cri-o://fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee" gracePeriod=30 Jul 22 17:02:18.697125 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:18.697007 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:18.931590 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:18.931562 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:19.266360 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.266324 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:19.366807 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.366773 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:19.592344 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.592318 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d"] Jul 22 17:02:19.592816 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.592731 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" Jul 22 17:02:19.592816 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.592749 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" Jul 22 17:02:19.592816 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.592774 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" Jul 22 17:02:19.592816 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.592781 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c606dea-1c2a-41d5-b971-cd5e4767e690" containerName="istio-proxy" Jul 22 17:02:19.593036 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.592858 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" Jul 22 17:02:19.593036 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.592869 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" Jul 22 17:02:19.595711 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.595691 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.598321 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.598301 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-265741-openshift-default-dockercfg-jww97\"" Jul 22 17:02:19.607742 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.607722 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d"] Jul 22 17:02:19.618021 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.617986 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:19.766906 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.766874 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-data\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.766906 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.766907 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-socket\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.767132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.766936 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szqrt\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-kube-api-access-szqrt\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.767132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.766986 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-credential-socket\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.767132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.767039 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-istio-token\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.767132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.767104 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-certs\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.767266 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.767144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da5f4384-6e22-4b37-b978-29763cdec194-istio-podinfo\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.767266 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.767164 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-envoy\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.767266 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.767189 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da5f4384-6e22-4b37-b978-29763cdec194-istiod-ca-cert\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868616 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868534 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-certs\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868616 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da5f4384-6e22-4b37-b978-29763cdec194-istio-podinfo\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868616 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868588 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-envoy\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868616 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da5f4384-6e22-4b37-b978-29763cdec194-istiod-ca-cert\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868943 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868636 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-data\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868943 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868664 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-socket\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868943 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868700 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szqrt\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-kube-api-access-szqrt\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868943 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868726 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-credential-socket\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.868943 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-istio-token\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.869202 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.868963 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-certs\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.869202 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.869120 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-socket\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.869202 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.869142 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-data\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.869322 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.869289 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-credential-socket\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.869721 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.869699 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da5f4384-6e22-4b37-b978-29763cdec194-istiod-ca-cert\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.871052 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.871026 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-envoy\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.871319 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.871300 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da5f4384-6e22-4b37-b978-29763cdec194-istio-podinfo\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.876417 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.876394 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-istio-token\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.876604 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.876585 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szqrt\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-kube-api-access-szqrt\") pod \"e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:19.905601 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:19.905575 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:20.024585 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:20.024542 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:20.038061 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:20.037990 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d"] Jul 22 17:02:20.040178 ip-10-0-138-200 kubenswrapper[2555]: W0722 17:02:20.040154 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5f4384_6e22_4b37_b978_29763cdec194.slice/crio-c9bd3c421eed21b080d014b2ecd9216295780ce28b8e04a7f3688a51e200784a WatchSource:0}: Error finding container c9bd3c421eed21b080d014b2ecd9216295780ce28b8e04a7f3688a51e200784a: Status 404 returned error can't find the container with id c9bd3c421eed21b080d014b2ecd9216295780ce28b8e04a7f3688a51e200784a Jul 22 17:02:20.042187 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:20.042158 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:20.042294 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:20.042230 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:20.042294 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:20.042278 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:21.043689 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:21.043648 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" event={"ID":"da5f4384-6e22-4b37-b978-29763cdec194","Type":"ContainerStarted","Data":"1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d"} Jul 22 17:02:21.044216 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:21.043700 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" event={"ID":"da5f4384-6e22-4b37-b978-29763cdec194","Type":"ContainerStarted","Data":"c9bd3c421eed21b080d014b2ecd9216295780ce28b8e04a7f3688a51e200784a"} Jul 22 17:02:21.064623 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:21.064562 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podStartSLOduration=2.064547672 podStartE2EDuration="2.064547672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:02:21.063285487 +0000 UTC m=+2383.768728042" watchObservedRunningTime="2026-07-22 17:02:21.064547672 +0000 UTC m=+2383.769990213" Jul 22 17:02:21.905872 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:21.905836 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:21.910596 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:21.910570 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:22.048150 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:22.048114 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:22.049044 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:22.049026 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:22.988474 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:22.988443 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:23.064617 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:23.064578 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:23.143243 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:23.143208 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:23.285747 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:23.285716 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:23.387492 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:23.387459 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:23.560517 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:23.560420 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:23.660579 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:23.660543 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:23.929167 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:23.929142 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:02:24.058975 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.058938 2555 generic.go:356] "Generic (PLEG): container finished" podID="83972a08-f752-460f-a95a-febaaef4a2b7" containerID="fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee" exitCode=0 Jul 22 17:02:24.059198 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.058992 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" Jul 22 17:02:24.059198 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.059019 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" event={"ID":"83972a08-f752-460f-a95a-febaaef4a2b7","Type":"ContainerDied","Data":"fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee"} Jul 22 17:02:24.059198 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.059069 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6" event={"ID":"83972a08-f752-460f-a95a-febaaef4a2b7","Type":"ContainerDied","Data":"56c1871d4d950973371a0a28dc89ecb407620d45f7217cad80a872696bf1d7c7"} Jul 22 17:02:24.059198 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.059126 2555 scope.go:117] "RemoveContainer" containerID="fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee" Jul 22 17:02:24.067738 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.067713 2555 scope.go:117] "RemoveContainer" containerID="6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713" Jul 22 17:02:24.075920 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.075896 2555 scope.go:117] "RemoveContainer" containerID="fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee" Jul 22 17:02:24.076360 ip-10-0-138-200 kubenswrapper[2555]: E0722 17:02:24.076338 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee\": container with ID starting with fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee not found: ID does not exist" containerID="fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee" Jul 22 17:02:24.076468 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.076370 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee"} err="failed to get container status \"fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee\": rpc error: code = NotFound desc = could not find container \"fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee\": container with ID starting with fd1cbae51eb390d5c26a932c708a746d83b6955735d581c3ab423fcefcb376ee not found: ID does not exist" Jul 22 17:02:24.076468 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.076391 2555 scope.go:117] "RemoveContainer" containerID="6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713" Jul 22 17:02:24.076643 ip-10-0-138-200 kubenswrapper[2555]: E0722 17:02:24.076629 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713\": container with ID starting with 6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713 not found: ID does not exist" containerID="6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713" Jul 22 17:02:24.076688 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.076647 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713"} err="failed to get container status \"6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713\": rpc error: code = NotFound desc = could not find container \"6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713\": container with ID starting with 6ea0af37e2e302755a9c3572af1ce4bd01265a269abb4c3651e4ff1f5aff1713 not found: ID does not exist" Jul 22 17:02:24.105842 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105753 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-data\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.105842 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105811 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-istio-token\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.105842 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105836 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-socket\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.106170 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105854 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-certs\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.106170 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105876 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8d6t\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-kube-api-access-s8d6t\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.106170 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105910 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-credential-socket\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.106170 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105949 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83972a08-f752-460f-a95a-febaaef4a2b7-istio-podinfo\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.106170 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105963 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-envoy\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.106170 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.105998 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83972a08-f752-460f-a95a-febaaef4a2b7-istiod-ca-cert\") pod \"83972a08-f752-460f-a95a-febaaef4a2b7\" (UID: \"83972a08-f752-460f-a95a-febaaef4a2b7\") " Jul 22 17:02:24.106170 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.106146 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:02:24.106554 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.106376 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:02:24.106554 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.106361 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:02:24.106554 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.106523 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83972a08-f752-460f-a95a-febaaef4a2b7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 17:02:24.107121 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.107078 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-data" (OuterVolumeSpecName: "istio-data") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:02:24.108372 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.108342 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83972a08-f752-460f-a95a-febaaef4a2b7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 17:02:24.108496 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.108383 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-istio-token" (OuterVolumeSpecName: "istio-token") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:02:24.108496 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.108404 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:02:24.108496 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.108432 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-kube-api-access-s8d6t" (OuterVolumeSpecName: "kube-api-access-s8d6t") pod "83972a08-f752-460f-a95a-febaaef4a2b7" (UID: "83972a08-f752-460f-a95a-febaaef4a2b7"). InnerVolumeSpecName "kube-api-access-s8d6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:02:24.206633 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206592 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206633 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206629 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206633 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206642 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206633 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206655 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8d6t\" (UniqueName: \"kubernetes.io/projected/83972a08-f752-460f-a95a-febaaef4a2b7-kube-api-access-s8d6t\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206997 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206668 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206997 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206685 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83972a08-f752-460f-a95a-febaaef4a2b7-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206997 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206697 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206997 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206711 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83972a08-f752-460f-a95a-febaaef4a2b7-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.206997 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.206723 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83972a08-f752-460f-a95a-febaaef4a2b7-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:02:24.379754 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.379722 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6"] Jul 22 17:02:24.383270 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:24.383246 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-828d6a-openshift-default-b686fc68c-sh6p6"] Jul 22 17:02:25.743546 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:25.743504 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" path="/var/lib/kubelet/pods/83972a08-f752-460f-a95a-febaaef4a2b7/volumes" Jul 22 17:02:27.511514 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:27.511473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:27.856198 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:27.856153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:28.139394 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139295 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr"] Jul 22 17:02:28.139705 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139691 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" Jul 22 17:02:28.139776 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139708 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" Jul 22 17:02:28.139776 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139728 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" Jul 22 17:02:28.139776 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139736 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" Jul 22 17:02:28.139776 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139751 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" Jul 22 17:02:28.139776 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139759 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f555c114-1a91-4f85-88d6-45c09502f094" containerName="istio-proxy" Jul 22 17:02:28.140388 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139822 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" Jul 22 17:02:28.140388 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.139833 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="83972a08-f752-460f-a95a-febaaef4a2b7" containerName="istio-proxy" Jul 22 17:02:28.144136 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.144113 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.146890 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.146864 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-13a8c9-openshift-default-dockercfg-wk9jk\"" Jul 22 17:02:28.154987 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.154954 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr"] Jul 22 17:02:28.173490 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.173440 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:28.340907 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.340855 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-credential-socket\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.340907 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.340912 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-data\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.341215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.340953 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/041d8182-3d5e-4f65-81c0-07b1444ce484-istiod-ca-cert\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.341215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.340983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-socket\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.341215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.341007 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-certs\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.341215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.341031 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7ql6\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-kube-api-access-p7ql6\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.341215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.341051 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-envoy\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.341215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.341072 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-podinfo\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.341215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.341141 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-token\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.404439 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.404326 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:28.441974 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.441937 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-data\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442178 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.441986 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/041d8182-3d5e-4f65-81c0-07b1444ce484-istiod-ca-cert\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442178 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442013 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-socket\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442178 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442039 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-certs\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442178 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442065 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7ql6\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-kube-api-access-p7ql6\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442410 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442298 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-envoy\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442410 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-podinfo\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442410 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442375 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-data\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442410 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442407 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-token\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442612 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442458 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-certs\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442612 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442526 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-credential-socket\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442612 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442555 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-socket\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442860 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442838 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-credential-socket\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.442898 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.442843 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/041d8182-3d5e-4f65-81c0-07b1444ce484-istiod-ca-cert\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.444589 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.444568 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-envoy\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.444851 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.444832 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-podinfo\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.449963 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.449942 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-token\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.450415 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.450399 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7ql6\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-kube-api-access-p7ql6\") pod \"e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.455274 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.455254 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:28.589274 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.589238 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr"] Jul 22 17:02:28.592114 ip-10-0-138-200 kubenswrapper[2555]: W0722 17:02:28.592069 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041d8182_3d5e_4f65_81c0_07b1444ce484.slice/crio-0286c472576518a05b080117a574fdab32afb30be43e264e547bc83cff7e7372 WatchSource:0}: Error finding container 0286c472576518a05b080117a574fdab32afb30be43e264e547bc83cff7e7372: Status 404 returned error can't find the container with id 0286c472576518a05b080117a574fdab32afb30be43e264e547bc83cff7e7372 Jul 22 17:02:28.594113 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.594057 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:28.594218 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.594156 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:28.594218 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.594197 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:28.993996 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:28.993956 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:02:29.081279 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:29.081241 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" event={"ID":"041d8182-3d5e-4f65-81c0-07b1444ce484","Type":"ContainerStarted","Data":"632b3e36c90f51a4476d23a6465838902c8bda7083d00f928a2d4349c27ae0da"} Jul 22 17:02:29.081461 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:29.081290 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" event={"ID":"041d8182-3d5e-4f65-81c0-07b1444ce484","Type":"ContainerStarted","Data":"0286c472576518a05b080117a574fdab32afb30be43e264e547bc83cff7e7372"} Jul 22 17:02:29.100696 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:29.100645 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" podStartSLOduration=1.100627246 podStartE2EDuration="1.100627246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:02:29.099502891 +0000 UTC m=+2391.804945435" watchObservedRunningTime="2026-07-22 17:02:29.100627246 +0000 UTC m=+2391.806069787" Jul 22 17:02:29.455461 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:29.455428 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:29.460990 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:29.460963 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:30.085682 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:30.085648 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:30.086757 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:30.086735 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:31.101848 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:31.101813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:31.182288 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:31.182242 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:31.254541 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:31.254508 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:31.640565 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:31.640529 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:31.754007 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:31.753976 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:31.852497 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:31.852460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:35.101428 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:35.101389 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:35.618759 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:35.618718 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:37.050143 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.050080 2555 patch_prober.go:28] interesting pod/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 22 17:02:37.050589 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.050164 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 22 17:02:37.118455 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.118410 2555 generic.go:356] "Generic (PLEG): container finished" podID="da5f4384-6e22-4b37-b978-29763cdec194" containerID="1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d" exitCode=0 Jul 22 17:02:37.118656 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.118479 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" event={"ID":"da5f4384-6e22-4b37-b978-29763cdec194","Type":"ContainerDied","Data":"1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d"} Jul 22 17:02:37.118963 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.118935 2555 scope.go:117] "RemoveContainer" containerID="1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d" Jul 22 17:02:37.120984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.120939 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:37.121112 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.121039 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:37.121189 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.121108 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:37.925076 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:37.925044 2555 scope.go:117] "RemoveContainer" containerID="803aa502e47150fbe7d27472c3a9174aac89d9ee371461623ec9737d83b76f61" Jul 22 17:02:38.126594 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:38.126557 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" event={"ID":"da5f4384-6e22-4b37-b978-29763cdec194","Type":"ContainerStarted","Data":"025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46"} Jul 22 17:02:38.907631 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:38.907544 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:38.907929 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:38.907903 2555 patch_prober.go:28] interesting pod/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 22 17:02:38.908106 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:38.908061 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 22 17:02:39.907450 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:39.906910 2555 patch_prober.go:28] interesting pod/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 22 17:02:39.907450 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:39.906994 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 22 17:02:40.907044 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:40.906996 2555 patch_prober.go:28] interesting pod/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 22 17:02:40.907265 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:40.907077 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 22 17:02:41.403752 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:41.403715 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:02:41.906363 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:41.906318 2555 patch_prober.go:28] interesting pod/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 22 17:02:41.906567 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:41.906393 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 22 17:02:42.911108 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:42.911069 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:42.911603 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:42.911497 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:42.912289 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:42.912272 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:02:43.149539 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:43.149506 2555 generic.go:356] "Generic (PLEG): container finished" podID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerID="632b3e36c90f51a4476d23a6465838902c8bda7083d00f928a2d4349c27ae0da" exitCode=0 Jul 22 17:02:43.149712 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:43.149589 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" event={"ID":"041d8182-3d5e-4f65-81c0-07b1444ce484","Type":"ContainerDied","Data":"632b3e36c90f51a4476d23a6465838902c8bda7083d00f928a2d4349c27ae0da"} Jul 22 17:02:43.150059 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:43.150044 2555 scope.go:117] "RemoveContainer" containerID="632b3e36c90f51a4476d23a6465838902c8bda7083d00f928a2d4349c27ae0da" Jul 22 17:02:43.150994 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:43.150970 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:43.151043 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:43.151032 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:43.151078 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:43.151062 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 17:02:44.155340 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:44.155298 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" event={"ID":"041d8182-3d5e-4f65-81c0-07b1444ce484","Type":"ContainerStarted","Data":"6d4ee440eeb3f014538930f18d4fa9538d3261fd93a1464a717df6183302bca6"} Jul 22 17:02:44.455782 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:44.455687 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:44.456887 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:44.456859 2555 patch_prober.go:28] interesting pod/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 22 17:02:44.457013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:44.456910 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 22 17:02:45.085605 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:45.085564 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:45.329782 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:45.329745 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:45.428810 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:45.428709 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:45.455755 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:45.455720 2555 patch_prober.go:28] interesting pod/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 22 17:02:45.455931 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:45.455789 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 22 17:02:46.456420 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:46.456381 2555 patch_prober.go:28] interesting pod/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 22 17:02:46.456821 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:46.456441 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 22 17:02:47.456385 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:47.456347 2555 patch_prober.go:28] interesting pod/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 22 17:02:47.456573 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:47.456402 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 22 17:02:48.330807 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:48.330755 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:48.456263 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:48.456221 2555 patch_prober.go:28] interesting pod/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 22 17:02:48.456469 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:48.456311 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 22 17:02:49.459706 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:49.459672 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:49.460774 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:49.460754 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:02:51.397110 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:51.397039 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:02:54.423029 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:54.422990 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:02:54.523411 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:02:54.523372 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:03:04.092875 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.092793 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k"] Jul 22 17:03:04.096105 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.096060 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.099981 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.099953 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-13a8c9\"/\"default-dockercfg-rlfst\"" Jul 22 17:03:04.100137 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.100045 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-13a8c9\"/\"auth-test-model-13a8c9-kserve-self-signed-certs\"" Jul 22 17:03:04.100137 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.099953 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-13a8c9\"/\"openshift-service-ca.crt\"" Jul 22 17:03:04.100137 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.100061 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-13a8c9\"/\"kube-root-ca.crt\"" Jul 22 17:03:04.106254 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.106234 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k"] Jul 22 17:03:04.143024 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.142995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:03:04.281312 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.281278 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-home\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.281312 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.281311 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tls-certs\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.281529 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.281386 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tmp-dir\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.281529 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.281460 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-dshm\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.281529 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.281508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hx97\" (UniqueName: \"kubernetes.io/projected/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-kube-api-access-5hx97\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.281631 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.281532 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-model-cache\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.382578 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tmp-dir\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.382578 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382542 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-dshm\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.382578 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382572 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hx97\" (UniqueName: \"kubernetes.io/projected/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-kube-api-access-5hx97\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.382800 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382592 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-model-cache\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.382800 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382609 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-home\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.382800 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382623 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tls-certs\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.382974 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382952 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tmp-dir\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.383043 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.382982 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-model-cache\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.383138 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.383112 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-home\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.384991 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.384971 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-dshm\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.385344 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.385327 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tls-certs\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.390029 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.390004 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hx97\" (UniqueName: \"kubernetes.io/projected/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-kube-api-access-5hx97\") pod \"auth-test-model-13a8c9-kserve-59b669875f-6z44k\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.405894 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.405873 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:04.532301 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:04.532195 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k"] Jul 22 17:03:04.534751 ip-10-0-138-200 kubenswrapper[2555]: W0722 17:03:04.534726 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb29e9c8f_72bf_48e2_9f70_0c61e66dcca7.slice/crio-1422fdcd27d0406c1ac1ab373239c0123bced1e428d21d86e71561e882beff28 WatchSource:0}: Error finding container 1422fdcd27d0406c1ac1ab373239c0123bced1e428d21d86e71561e882beff28: Status 404 returned error can't find the container with id 1422fdcd27d0406c1ac1ab373239c0123bced1e428d21d86e71561e882beff28 Jul 22 17:03:05.244969 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:05.244931 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" event={"ID":"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7","Type":"ContainerStarted","Data":"0312a8db91094174d5b2e232f72fcefba80f3b0bfa26ef118533c588b32618a9"} Jul 22 17:03:05.244969 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:05.244974 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" event={"ID":"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7","Type":"ContainerStarted","Data":"1422fdcd27d0406c1ac1ab373239c0123bced1e428d21d86e71561e882beff28"} Jul 22 17:03:05.263758 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:05.263706 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" podStartSLOduration=1.263691842 podStartE2EDuration="1.263691842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:03:05.262160314 +0000 UTC m=+2427.967602857" watchObservedRunningTime="2026-07-22 17:03:05.263691842 +0000 UTC m=+2427.969134383" Jul 22 17:03:07.991208 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:07.991169 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:03:14.406786 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:14.406741 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:14.407220 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:14.406807 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:14.419590 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:14.419561 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:15.296508 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:15.296480 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:03:28.185220 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:28.185177 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:03:28.263279 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:28.263231 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:03:29.787355 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:29.787320 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:03:47.367522 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:03:47.367487 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:04:01.252707 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:04:01.252660 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:04:01.306776 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:04:01.306738 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:04:05.589203 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:04:05.589168 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:04:25.103860 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:04:25.103820 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:04:33.478666 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:04:33.478627 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:04:44.795456 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:04:44.795419 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:05:03.398299 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:03.398252 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:05:18.814748 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:18.814713 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:05:19.720421 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:19.720387 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k"] Jul 22 17:05:20.576488 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:20.576451 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:05:23.493629 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:23.493590 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:05:33.623852 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:33.623814 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:05:34.741836 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:34.741794 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" podUID="b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" containerName="main" containerID="cri-o://0312a8db91094174d5b2e232f72fcefba80f3b0bfa26ef118533c588b32618a9" gracePeriod=45 Jul 22 17:05:34.860132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:34.860074 2555 generic.go:356] "Generic (PLEG): container finished" podID="b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" containerID="0312a8db91094174d5b2e232f72fcefba80f3b0bfa26ef118533c588b32618a9" exitCode=0 Jul 22 17:05:34.860132 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:34.860119 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" event={"ID":"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7","Type":"ContainerDied","Data":"0312a8db91094174d5b2e232f72fcefba80f3b0bfa26ef118533c588b32618a9"} Jul 22 17:05:34.998060 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:34.997988 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:05:35.050932 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.050895 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hx97\" (UniqueName: \"kubernetes.io/projected/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-kube-api-access-5hx97\") pod \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " Jul 22 17:05:35.051145 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.050978 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tls-certs\") pod \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " Jul 22 17:05:35.051145 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.051019 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-home\") pod \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " Jul 22 17:05:35.051145 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.051045 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tmp-dir\") pod \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " Jul 22 17:05:35.051145 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.051062 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-dshm\") pod \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " Jul 22 17:05:35.051145 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.051111 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-model-cache\") pod \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\" (UID: \"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7\") " Jul 22 17:05:35.051432 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.051350 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-home" (OuterVolumeSpecName: "home") pod "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" (UID: "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:05:35.051432 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.051354 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" (UID: "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:05:35.051432 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.051380 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-model-cache" (OuterVolumeSpecName: "model-cache") pod "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" (UID: "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:05:35.053356 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.053331 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" (UID: "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 17:05:35.053356 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.053345 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-kube-api-access-5hx97" (OuterVolumeSpecName: "kube-api-access-5hx97") pod "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" (UID: "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7"). InnerVolumeSpecName "kube-api-access-5hx97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:05:35.053356 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.053334 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-dshm" (OuterVolumeSpecName: "dshm") pod "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" (UID: "b29e9c8f-72bf-48e2-9f70-0c61e66dcca7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:05:35.152175 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.152135 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tls-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:05:35.152175 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.152167 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-home\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:05:35.152175 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.152176 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-tmp-dir\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:05:35.152175 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.152182 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-dshm\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:05:35.152175 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.152191 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-model-cache\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:05:35.152475 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.152199 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hx97\" (UniqueName: \"kubernetes.io/projected/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7-kube-api-access-5hx97\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:05:35.870648 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.870609 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" event={"ID":"b29e9c8f-72bf-48e2-9f70-0c61e66dcca7","Type":"ContainerDied","Data":"1422fdcd27d0406c1ac1ab373239c0123bced1e428d21d86e71561e882beff28"} Jul 22 17:05:35.871156 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.870659 2555 scope.go:117] "RemoveContainer" containerID="0312a8db91094174d5b2e232f72fcefba80f3b0bfa26ef118533c588b32618a9" Jul 22 17:05:35.871156 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.870661 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k" Jul 22 17:05:35.886810 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.886781 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k"] Jul 22 17:05:35.890835 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:35.890810 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-13a8c9/auth-test-model-13a8c9-kserve-59b669875f-6z44k"] Jul 22 17:05:36.896416 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:36.896386 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:05:37.743924 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:37.743885 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" path="/var/lib/kubelet/pods/b29e9c8f-72bf-48e2-9f70-0c61e66dcca7/volumes" Jul 22 17:05:37.833007 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:37.832951 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:05:37.934131 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:37.934097 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:05:53.515958 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:53.515927 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:05:57.197802 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:05:57.197765 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:06:11.694946 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:11.694867 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:06:28.258920 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:28.258859 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:06:29.194034 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:29.194003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:06:33.191070 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:33.191024 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:06:43.722210 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:43.722172 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:06:46.642065 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:46.642027 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:06:54.199987 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:54.199943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:06:54.297779 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:06:54.297742 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:07:13.805724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:13.805671 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:07:17.399684 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:17.399649 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:07:39.591523 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:39.591483 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:07:46.032403 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:46.032338 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:07:46.133438 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:46.133402 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:07:54.401402 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:54.401365 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:07:55.168037 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:55.168004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:07:55.269731 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:07:55.269698 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:01.233403 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:01.233368 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:02.197013 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:02.196974 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:13.992678 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:13.992642 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:08:21.593523 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:21.593484 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:21.692504 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:21.692463 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:31.612945 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:31.612909 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:34.699359 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:34.699325 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:08:36.142505 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:36.142466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:36.548647 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:36.548601 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:42.493049 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.492994 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:42.558572 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.558536 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:42.724624 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.724590 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d"] Jul 22 17:08:42.724860 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.724821 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" containerID="cri-o://025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46" gracePeriod=30 Jul 22 17:08:42.747606 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.747510 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:42.913593 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.913560 2555 patch_prober.go:28] interesting pod/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 17:08:42.913761 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.913626 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 17:08:42.968945 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:42.968913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:43.620441 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:43.620401 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:43.724986 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:43.724947 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:47.977253 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:47.977223 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:08:48.078148 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078117 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da5f4384-6e22-4b37-b978-29763cdec194-istio-podinfo\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078332 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078182 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-envoy\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078332 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078204 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-credential-socket\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078332 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078220 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-istio-token\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078332 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078266 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da5f4384-6e22-4b37-b978-29763cdec194-istiod-ca-cert\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078332 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078307 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szqrt\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-kube-api-access-szqrt\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078609 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078337 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-data\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078609 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078360 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-certs\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078609 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078428 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-socket\") pod \"da5f4384-6e22-4b37-b978-29763cdec194\" (UID: \"da5f4384-6e22-4b37-b978-29763cdec194\") " Jul 22 17:08:48.078609 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078528 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:08:48.078813 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078722 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.078813 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078722 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:08:48.078914 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078883 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:08:48.078914 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.078890 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da5f4384-6e22-4b37-b978-29763cdec194-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 17:08:48.079769 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.079736 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-data" (OuterVolumeSpecName: "istio-data") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:08:48.080596 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.080572 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:08:48.080801 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.080777 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-kube-api-access-szqrt" (OuterVolumeSpecName: "kube-api-access-szqrt") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "kube-api-access-szqrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:08:48.080988 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.080962 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-istio-token" (OuterVolumeSpecName: "istio-token") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:08:48.080988 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.080962 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da5f4384-6e22-4b37-b978-29763cdec194-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da5f4384-6e22-4b37-b978-29763cdec194" (UID: "da5f4384-6e22-4b37-b978-29763cdec194"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 17:08:48.180159 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180120 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szqrt\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-kube-api-access-szqrt\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.180159 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180151 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.180159 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180161 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.180159 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180170 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.180422 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180179 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da5f4384-6e22-4b37-b978-29763cdec194-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.180422 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180187 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da5f4384-6e22-4b37-b978-29763cdec194-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.180422 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180194 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da5f4384-6e22-4b37-b978-29763cdec194-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.180422 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.180201 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da5f4384-6e22-4b37-b978-29763cdec194-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:08:48.645176 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.645140 2555 generic.go:356] "Generic (PLEG): container finished" podID="da5f4384-6e22-4b37-b978-29763cdec194" containerID="025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46" exitCode=0 Jul 22 17:08:48.645321 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.645191 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" Jul 22 17:08:48.645321 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.645197 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" event={"ID":"da5f4384-6e22-4b37-b978-29763cdec194","Type":"ContainerDied","Data":"025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46"} Jul 22 17:08:48.645321 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.645239 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d" event={"ID":"da5f4384-6e22-4b37-b978-29763cdec194","Type":"ContainerDied","Data":"c9bd3c421eed21b080d014b2ecd9216295780ce28b8e04a7f3688a51e200784a"} Jul 22 17:08:48.645321 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.645260 2555 scope.go:117] "RemoveContainer" containerID="025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46" Jul 22 17:08:48.654144 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.654079 2555 scope.go:117] "RemoveContainer" containerID="1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d" Jul 22 17:08:48.662064 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.662044 2555 scope.go:117] "RemoveContainer" containerID="025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46" Jul 22 17:08:48.662354 ip-10-0-138-200 kubenswrapper[2555]: E0722 17:08:48.662331 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46\": container with ID starting with 025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46 not found: ID does not exist" containerID="025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46" Jul 22 17:08:48.662433 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.662366 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46"} err="failed to get container status \"025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46\": rpc error: code = NotFound desc = could not find container \"025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46\": container with ID starting with 025d13382156457dfbbc324b606517a63243f69ef06236cfd1f9e34f6a564a46 not found: ID does not exist" Jul 22 17:08:48.662433 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.662392 2555 scope.go:117] "RemoveContainer" containerID="1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d" Jul 22 17:08:48.662641 ip-10-0-138-200 kubenswrapper[2555]: E0722 17:08:48.662622 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d\": container with ID starting with 1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d not found: ID does not exist" containerID="1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d" Jul 22 17:08:48.662682 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.662647 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d"} err="failed to get container status \"1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d\": rpc error: code = NotFound desc = could not find container \"1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d\": container with ID starting with 1cd17be39bf719e18829e242639e78178b58934a765fd00633de1c878a2f890d not found: ID does not exist" Jul 22 17:08:48.666743 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.666718 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d"] Jul 22 17:08:48.670653 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:48.670632 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-265741-openshift-default-85d8c78b79-nbc6d"] Jul 22 17:08:49.743842 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:49.743808 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da5f4384-6e22-4b37-b978-29763cdec194" path="/var/lib/kubelet/pods/da5f4384-6e22-4b37-b978-29763cdec194/volumes" Jul 22 17:08:53.503308 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:53.503273 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:56.493905 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:56.493868 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xvsmd"] Jul 22 17:08:58.179895 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:58.179860 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:08:58.541321 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:08:58.541279 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:04.296611 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:04.296576 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:04.396361 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:04.396322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:04.601418 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:04.601332 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr"] Jul 22 17:09:04.601570 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:04.601511 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" containerID="cri-o://6d4ee440eeb3f014538930f18d4fa9538d3261fd93a1464a717df6183302bca6" gracePeriod=30 Jul 22 17:09:04.613806 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:04.613781 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:04.803647 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:04.803614 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:09.727789 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.727750 2555 generic.go:356] "Generic (PLEG): container finished" podID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerID="6d4ee440eeb3f014538930f18d4fa9538d3261fd93a1464a717df6183302bca6" exitCode=0 Jul 22 17:09:09.728174 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.727824 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" event={"ID":"041d8182-3d5e-4f65-81c0-07b1444ce484","Type":"ContainerDied","Data":"6d4ee440eeb3f014538930f18d4fa9538d3261fd93a1464a717df6183302bca6"} Jul 22 17:09:09.728174 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.727916 2555 scope.go:117] "RemoveContainer" containerID="632b3e36c90f51a4476d23a6465838902c8bda7083d00f928a2d4349c27ae0da" Jul 22 17:09:09.852531 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.852506 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:09:09.946505 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946413 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-token\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946505 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946492 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-socket\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946513 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7ql6\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-kube-api-access-p7ql6\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946530 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-envoy\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946550 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-podinfo\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946564 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-certs\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946591 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/041d8182-3d5e-4f65-81c0-07b1444ce484-istiod-ca-cert\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946661 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-credential-socket\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.946724 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946700 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-data\") pod \"041d8182-3d5e-4f65-81c0-07b1444ce484\" (UID: \"041d8182-3d5e-4f65-81c0-07b1444ce484\") " Jul 22 17:09:09.947115 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.946877 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:09:09.947192 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.947113 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:09:09.947248 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.947218 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/041d8182-3d5e-4f65-81c0-07b1444ce484-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 17:09:09.947331 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.947311 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:09:09.947600 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.947579 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-data" (OuterVolumeSpecName: "istio-data") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:09:09.949235 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.949215 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-token" (OuterVolumeSpecName: "istio-token") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:09:09.949322 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.949252 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-kube-api-access-p7ql6" (OuterVolumeSpecName: "kube-api-access-p7ql6") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "kube-api-access-p7ql6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:09:09.949362 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.949314 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:09:09.949362 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:09.949314 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "041d8182-3d5e-4f65-81c0-07b1444ce484" (UID: "041d8182-3d5e-4f65-81c0-07b1444ce484"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 17:09:10.047797 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047760 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-credential-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.047797 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047792 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-data\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.047797 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047804 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-token\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.048055 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047815 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-socket\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.048055 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047827 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7ql6\" (UniqueName: \"kubernetes.io/projected/041d8182-3d5e-4f65-81c0-07b1444ce484-kube-api-access-p7ql6\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.048055 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047839 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-envoy\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.048055 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047849 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/041d8182-3d5e-4f65-81c0-07b1444ce484-istio-podinfo\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.048055 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047859 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/041d8182-3d5e-4f65-81c0-07b1444ce484-workload-certs\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.048055 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.047870 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/041d8182-3d5e-4f65-81c0-07b1444ce484-istiod-ca-cert\") on node \"ip-10-0-138-200.ec2.internal\" DevicePath \"\"" Jul 22 17:09:10.735276 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.735242 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" event={"ID":"041d8182-3d5e-4f65-81c0-07b1444ce484","Type":"ContainerDied","Data":"0286c472576518a05b080117a574fdab32afb30be43e264e547bc83cff7e7372"} Jul 22 17:09:10.735693 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.735298 2555 scope.go:117] "RemoveContainer" containerID="6d4ee440eeb3f014538930f18d4fa9538d3261fd93a1464a717df6183302bca6" Jul 22 17:09:10.735693 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.735260 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr" Jul 22 17:09:10.757469 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.757443 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr"] Jul 22 17:09:10.761118 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:10.761080 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-13a8c9-openshift-default-b7c6fdcdc-9vwpr"] Jul 22 17:09:11.227215 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:11.227182 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b566c8c6-wcbdd_25a6dfc5-f101-4f84-a7b3-20e7c786e615/manager/0.log" Jul 22 17:09:11.232828 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:11.232806 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b566c8c6-wcbdd_25a6dfc5-f101-4f84-a7b3-20e7c786e615/copy-manifests/0.log" Jul 22 17:09:11.692325 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:11.692295 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745645-fsr9g_a6912381-5d4e-4370-a408-0a62beab2bd7/cleanup/0.log" Jul 22 17:09:11.743934 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:11.743896 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" path="/var/lib/kubelet/pods/041d8182-3d5e-4f65-81c0-07b1444ce484/volumes" Jul 22 17:09:13.852783 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:13.852749 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xvsmd_e7aa9cf6-5081-4c71-be7c-1dbae4c303c6/limitador/0.log" Jul 22 17:09:14.307728 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:14.307696 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vgfnj_ca0de617-2373-42f4-928e-981572640e95/istio-proxy/0.log" Jul 22 17:09:14.430557 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:14.430517 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-94nwx_50046a88-7376-4b4b-a4b3-fdb5f1ca989f/discovery/0.log" Jul 22 17:09:21.308580 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308545 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s2qjm/must-gather-2lwzb"] Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308800 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308810 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308817 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308822 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308838 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308844 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308849 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" containerName="main" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308854 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" containerName="main" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308865 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308870 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308918 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308926 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308931 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="041d8182-3d5e-4f65-81c0-07b1444ce484" containerName="istio-proxy" Jul 22 17:09:21.308984 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.308937 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b29e9c8f-72bf-48e2-9f70-0c61e66dcca7" containerName="main" Jul 22 17:09:21.309443 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.309026 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="da5f4384-6e22-4b37-b978-29763cdec194" containerName="istio-proxy" Jul 22 17:09:21.311802 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.311781 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.314382 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.314359 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s2qjm\"/\"kube-root-ca.crt\"" Jul 22 17:09:21.314491 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.314435 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s2qjm\"/\"default-dockercfg-2dmcn\"" Jul 22 17:09:21.315393 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.315379 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s2qjm\"/\"openshift-service-ca.crt\"" Jul 22 17:09:21.320774 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.320754 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2qjm/must-gather-2lwzb"] Jul 22 17:09:21.330548 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.330507 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:21.429793 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.429754 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:21.435788 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.435758 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0cf22627-0266-463e-b49f-cdcbf879b8e6-must-gather-output\") pod \"must-gather-2lwzb\" (UID: \"0cf22627-0266-463e-b49f-cdcbf879b8e6\") " pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.435871 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.435812 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpsv\" (UniqueName: \"kubernetes.io/projected/0cf22627-0266-463e-b49f-cdcbf879b8e6-kube-api-access-5kpsv\") pod \"must-gather-2lwzb\" (UID: \"0cf22627-0266-463e-b49f-cdcbf879b8e6\") " pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.536817 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.536784 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0cf22627-0266-463e-b49f-cdcbf879b8e6-must-gather-output\") pod \"must-gather-2lwzb\" (UID: \"0cf22627-0266-463e-b49f-cdcbf879b8e6\") " pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.536942 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.536838 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpsv\" (UniqueName: \"kubernetes.io/projected/0cf22627-0266-463e-b49f-cdcbf879b8e6-kube-api-access-5kpsv\") pod \"must-gather-2lwzb\" (UID: \"0cf22627-0266-463e-b49f-cdcbf879b8e6\") " pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.537160 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.537142 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0cf22627-0266-463e-b49f-cdcbf879b8e6-must-gather-output\") pod \"must-gather-2lwzb\" (UID: \"0cf22627-0266-463e-b49f-cdcbf879b8e6\") " pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.546042 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.546025 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpsv\" (UniqueName: \"kubernetes.io/projected/0cf22627-0266-463e-b49f-cdcbf879b8e6-kube-api-access-5kpsv\") pod \"must-gather-2lwzb\" (UID: \"0cf22627-0266-463e-b49f-cdcbf879b8e6\") " pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.621710 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.621621 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2qjm/must-gather-2lwzb" Jul 22 17:09:21.749803 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.749772 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2qjm/must-gather-2lwzb"] Jul 22 17:09:21.752875 ip-10-0-138-200 kubenswrapper[2555]: W0722 17:09:21.752836 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cf22627_0266_463e_b49f_cdcbf879b8e6.slice/crio-94263f966e5a2beaa7840f26a812a14d4264c4797a622355ad8d0f35ce5166e1 WatchSource:0}: Error finding container 94263f966e5a2beaa7840f26a812a14d4264c4797a622355ad8d0f35ce5166e1: Status 404 returned error can't find the container with id 94263f966e5a2beaa7840f26a812a14d4264c4797a622355ad8d0f35ce5166e1 Jul 22 17:09:21.754695 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.754679 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 17:09:21.781697 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:21.781671 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2qjm/must-gather-2lwzb" event={"ID":"0cf22627-0266-463e-b49f-cdcbf879b8e6","Type":"ContainerStarted","Data":"94263f966e5a2beaa7840f26a812a14d4264c4797a622355ad8d0f35ce5166e1"} Jul 22 17:09:22.788559 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:22.788468 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2qjm/must-gather-2lwzb" event={"ID":"0cf22627-0266-463e-b49f-cdcbf879b8e6","Type":"ContainerStarted","Data":"6c057ff5bcf570340a3724d3db3c4c943d2ccf9caa120b3f6bef2d6ad4a9ff3f"} Jul 22 17:09:22.788559 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:22.788516 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2qjm/must-gather-2lwzb" event={"ID":"0cf22627-0266-463e-b49f-cdcbf879b8e6","Type":"ContainerStarted","Data":"90430f6ff5f23dc69b3eaeacca27b78bd847a3f66b0a03cb38eab0048c5c14db"} Jul 22 17:09:22.811198 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:22.811135 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s2qjm/must-gather-2lwzb" podStartSLOduration=1.025738242 podStartE2EDuration="1.811077416s" totalImagesPullingTime="785.339174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:09:22.809453715 +0000 UTC m=+2805.514896257" watchObservedRunningTime="2026-07-22 17:09:22.811077416 +0000 UTC m=+2805.516519959" Jul 22 17:09:24.073286 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:24.073252 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ckfjr_df349bb5-3a16-47f3-80dc-4d13678538c2/global-pull-secret-syncer/0.log" Jul 22 17:09:24.211397 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:24.211366 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" Jul 22 17:09:24.318430 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:24.318396 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-200.ec2.internal_a7300a218e12596f33fabcbb2467c251/haproxy/0.log" Jul 22 17:09:29.837579 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:29.837551 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xvsmd_e7aa9cf6-5081-4c71-be7c-1dbae4c303c6/limitador/0.log" Jul 22 17:09:31.926153 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:31.926122 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2ctm_9508b4dc-16a7-4705-b35a-c85e0df8202d/node-exporter/0.log" Jul 22 17:09:31.949438 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:31.949404 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2ctm_9508b4dc-16a7-4705-b35a-c85e0df8202d/kube-rbac-proxy/0.log" Jul 22 17:09:31.971631 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:31.971602 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2ctm_9508b4dc-16a7-4705-b35a-c85e0df8202d/init-textfile/0.log" Jul 22 17:09:34.077687 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.077647 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx"] Jul 22 17:09:34.083006 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.082981 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.089019 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.088991 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx"] Jul 22 17:09:34.158945 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.158912 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-podres\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.159171 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.158984 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-proc\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.159171 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.159018 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-sys\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.159171 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.159105 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwv8w\" (UniqueName: \"kubernetes.io/projected/f53e3e27-398a-463d-b371-9bf416ca06f6-kube-api-access-nwv8w\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.159171 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.159138 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-lib-modules\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.259884 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.259840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-podres\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260158 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260065 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-podres\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260410 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260369 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-proc\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260496 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-sys\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260496 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260484 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwv8w\" (UniqueName: \"kubernetes.io/projected/f53e3e27-398a-463d-b371-9bf416ca06f6-kube-api-access-nwv8w\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260625 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-lib-modules\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260744 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260726 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-lib-modules\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260863 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260841 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-sys\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.260989 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.260973 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f53e3e27-398a-463d-b371-9bf416ca06f6-proc\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.271281 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.271247 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwv8w\" (UniqueName: \"kubernetes.io/projected/f53e3e27-398a-463d-b371-9bf416ca06f6-kube-api-access-nwv8w\") pod \"perf-node-gather-daemonset-hxnqx\" (UID: \"f53e3e27-398a-463d-b371-9bf416ca06f6\") " pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.399390 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.399307 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:34.595075 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.595005 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx"] Jul 22 17:09:34.858505 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.858462 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" event={"ID":"f53e3e27-398a-463d-b371-9bf416ca06f6","Type":"ContainerStarted","Data":"004b6b1846506eb3c77b3eb6fe1e2a1c8f421bc92f1418e6b1f58077a0ff7926"} Jul 22 17:09:34.858700 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.858528 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" event={"ID":"f53e3e27-398a-463d-b371-9bf416ca06f6","Type":"ContainerStarted","Data":"41c1db7e0c390ab79976e763dd22d5da931952710dc88c9dd98af26c2710678a"} Jul 22 17:09:34.858700 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:34.858554 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:36.669775 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:36.669745 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g6gmg_46d4b72d-b63d-4e54-a782-abfdd53f3c48/dns/0.log" Jul 22 17:09:36.691558 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:36.691527 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g6gmg_46d4b72d-b63d-4e54-a782-abfdd53f3c48/kube-rbac-proxy/0.log" Jul 22 17:09:36.816568 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:36.816535 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-szwzs_9090613d-21a5-4fd0-a284-6503d8fe777e/dns-node-resolver/0.log" Jul 22 17:09:37.360320 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:37.360281 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5c485f5fc9-j6rf8_77539c43-325f-46a5-bed1-7a78c000a7e7/registry/0.log" Jul 22 17:09:37.384549 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:37.384520 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pvvg_7e4879e5-9f03-4efa-8875-01f04f0efe05/node-ca/0.log" Jul 22 17:09:38.384847 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:38.384809 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vgfnj_ca0de617-2373-42f4-928e-981572640e95/istio-proxy/0.log" Jul 22 17:09:38.619908 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:38.619872 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-94nwx_50046a88-7376-4b4b-a4b3-fdb5f1ca989f/discovery/0.log" Jul 22 17:09:39.496958 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:39.496920 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7mrnv_0618f128-da4f-48f9-80f8-2556651f28e4/serve-healthcheck-canary/0.log" Jul 22 17:09:40.135531 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:40.135480 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qvpmq_abbf1ead-2639-429c-8a3b-eb8664f3996d/kube-rbac-proxy/0.log" Jul 22 17:09:40.156985 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:40.156949 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qvpmq_abbf1ead-2639-429c-8a3b-eb8664f3996d/exporter/0.log" Jul 22 17:09:40.182648 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:40.182614 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qvpmq_abbf1ead-2639-429c-8a3b-eb8664f3996d/extractor/0.log" Jul 22 17:09:40.878534 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:40.878496 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" Jul 22 17:09:40.897670 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:40.897622 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s2qjm/perf-node-gather-daemonset-hxnqx" podStartSLOduration=6.897607766 podStartE2EDuration="6.897607766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 17:09:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:09:34.876598357 +0000 UTC m=+2817.582040900" watchObservedRunningTime="2026-07-22 17:09:40.897607766 +0000 UTC m=+2823.603050307" Jul 22 17:09:42.652917 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:42.652884 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-49m6s_12c56210-7d70-45ed-bb55-7ca81950c8f4/konnectivity-agent/0.log" limit=1024 Jul 22 17:09:42.811329 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:42.811296 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b566c8c6-wcbdd_25a6dfc5-f101-4f84-a7b3-20e7c786e615/manager/0.log" Jul 22 17:09:42.831316 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:42.831286 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b566c8c6-wcbdd_25a6dfc5-f101-4f84-a7b3-20e7c786e615/copy-manifests/0.log" Jul 22 17:09:44.287580 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:44.287550 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65dd5d6b6d-b68tr_7c4a9d28-c931-441f-8fcf-a5caf5607b5d/manager/0.log" Jul 22 17:09:44.660486 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:44.660457 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vgfnj_ca0de617-2373-42f4-928e-981572640e95/istio-proxy/0.log" Jul 22 17:09:44.868418 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:44.868374 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-94nwx_50046a88-7376-4b4b-a4b3-fdb5f1ca989f/discovery/0.log" Jul 22 17:09:46.203618 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:46.203560 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vgfnj_ca0de617-2373-42f4-928e-981572640e95/istio-proxy/0.log" Jul 22 17:09:46.421019 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:46.420980 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-94nwx_50046a88-7376-4b4b-a4b3-fdb5f1ca989f/discovery/0.log" Jul 22 17:09:49.247464 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:49.247433 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ndvc8_24e58fae-3f60-4a62-ae19-2d3a7396628c/migrator/0.log" Jul 22 17:09:49.267697 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:49.267665 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ndvc8_24e58fae-3f60-4a62-ae19-2d3a7396628c/graceful-termination/0.log" Jul 22 17:09:50.626879 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.626852 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-99zkh_29bfdb7e-ccfa-4f28-9958-fd4281974e45/kube-multus/0.log" Jul 22 17:09:50.843634 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.843604 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2wx6_f7b166af-2797-4f86-b009-667c2d19b6b7/kube-multus-additional-cni-plugins/0.log" Jul 22 17:09:50.864409 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.864374 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2wx6_f7b166af-2797-4f86-b009-667c2d19b6b7/egress-router-binary-copy/0.log" Jul 22 17:09:50.884335 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.884267 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2wx6_f7b166af-2797-4f86-b009-667c2d19b6b7/cni-plugins/0.log" Jul 22 17:09:50.903523 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.903485 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2wx6_f7b166af-2797-4f86-b009-667c2d19b6b7/bond-cni-plugin/0.log" Jul 22 17:09:50.924465 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.924441 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2wx6_f7b166af-2797-4f86-b009-667c2d19b6b7/routeoverride-cni/0.log" Jul 22 17:09:50.943526 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.943504 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2wx6_f7b166af-2797-4f86-b009-667c2d19b6b7/whereabouts-cni-bincopy/0.log" Jul 22 17:09:50.963621 ip-10-0-138-200 kubenswrapper[2555]: I0722 17:09:50.963598 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n2wx6_f7b166af-2797-4f86-b009-667c2d19b6b7/whereabouts-cni/0.log"