Jul 07 19:59:04.371165 ip-10-0-133-86 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 19:59:04.371176 ip-10-0-133-86 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 19:59:04.371186 ip-10-0-133-86 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 19:59:04.371498 ip-10-0-133-86 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 19:59:14.386613 ip-10-0-133-86 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 19:59:14.386630 ip-10-0-133-86 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1246dd5cd5404e61b2ff696f67407330 -- Jul 07 20:00:46.064287 ip-10-0-133-86 systemd[1]: Starting Kubernetes Kubelet... Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.519220 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524938 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524949 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524953 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524956 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524961 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524963 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524966 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 20:00:46.530505 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524969 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524972 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524975 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524977 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524980 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524984 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524986 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524989 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524992 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524995 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.524998 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525001 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525003 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525007 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525009 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525013 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525036 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525039 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525042 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 20:00:46.545164 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525045 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525047 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525051 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525054 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525058 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525063 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525067 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525070 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525073 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525075 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525078 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525081 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525084 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525086 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525089 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525092 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525095 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525098 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525101 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 20:00:46.546954 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525104 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525108 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525111 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525115 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525118 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525120 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525123 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525126 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525129 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525131 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525134 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525136 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525139 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525141 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525144 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525146 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525149 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525151 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525154 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525156 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 20:00:46.549259 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525159 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525161 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525163 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525166 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525168 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525171 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525173 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525176 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525179 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525181 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525184 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525187 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525190 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525193 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525196 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525199 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525201 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525204 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525206 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 20:00:46.564932 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525209 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525212 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525215 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525218 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525220 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525222 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525225 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525228 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525230 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525233 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525235 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525238 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525240 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525243 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525247 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525926 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525932 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525936 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525939 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525942 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 20:00:46.567551 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525945 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525948 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525952 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525955 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525958 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525961 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525963 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525966 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525970 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525972 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525975 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525977 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525980 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525983 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525987 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525990 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525993 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525996 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.525999 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 20:00:46.569443 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526002 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526004 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526008 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526010 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526013 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526031 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526034 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526036 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526039 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526042 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526045 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526047 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526051 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526054 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526057 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526059 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526062 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526065 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526068 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526070 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 20:00:46.571872 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526074 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526078 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526081 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526084 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526087 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526090 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526095 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526098 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526100 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526103 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526105 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526108 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526110 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526113 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526116 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526118 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526121 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526123 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526126 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526128 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 20:00:46.584915 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526131 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526134 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526136 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526139 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526141 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526144 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526147 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526149 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526152 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526154 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526157 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526160 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526163 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526165 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526168 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526171 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526174 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526176 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526180 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 20:00:46.587533 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526182 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526185 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526187 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526190 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526192 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526197 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526200 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526203 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526205 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526208 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526210 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526213 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526216 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526218 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526221 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526223 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526577 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526581 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526584 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 20:00:46.589414 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526587 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526590 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526593 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526595 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526598 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526601 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526603 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526606 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526608 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526612 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526615 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526618 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526622 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526625 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526627 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526630 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526633 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526635 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526639 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526641 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 20:00:46.597322 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526644 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526646 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526649 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526652 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526654 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526657 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526659 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526662 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526664 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526666 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526669 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526671 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526674 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526676 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526679 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526681 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526684 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526687 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526691 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526693 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 20:00:46.599556 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526695 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526698 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526701 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526703 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526707 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526709 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526712 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526715 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526717 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526720 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526725 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526728 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526730 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526733 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526736 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526739 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526742 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526744 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526747 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 20:00:46.601946 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526749 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526752 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526754 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526758 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526760 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526762 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526765 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526768 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526770 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526773 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526776 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526778 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526781 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526783 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526785 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526788 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526792 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526797 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526800 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526802 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 20:00:46.603821 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526805 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 20:00:46.603617 ip-10-0-133-86 systemd[1]: Started Kubernetes Kubelet. Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526807 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526810 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526814 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526816 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526819 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526821 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526824 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526826 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526829 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526831 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526834 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526836 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526839 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526841 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526844 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.526846 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526912 2553 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526919 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526924 2553 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526928 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 20:00:46.618421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526932 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526935 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526939 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526942 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526946 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526949 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526952 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526955 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526960 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526963 2553 flags.go:64] FLAG: --cgroup-root="" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526966 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526969 2553 flags.go:64] FLAG: --client-ca-file="" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526972 2553 flags.go:64] FLAG: --cloud-provider="external" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.526975 2553 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527001 2553 flags.go:64] FLAG: --cluster-domain="" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527004 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527008 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527011 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527027 2553 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527031 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527034 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527037 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527041 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527043 2553 flags.go:64] FLAG: --contention-profiling="false" Jul 07 20:00:46.634127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527046 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527049 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527052 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527055 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527059 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527062 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527065 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527068 2553 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527071 2553 flags.go:64] FLAG: --enable-server="true" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527073 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527077 2553 flags.go:64] FLAG: --event-burst="100" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527080 2553 flags.go:64] FLAG: --event-qps="50" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527083 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527086 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527089 2553 flags.go:64] FLAG: --eviction-hard="" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527092 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527097 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527100 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527103 2553 flags.go:64] FLAG: --eviction-soft="" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527106 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527109 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527112 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527116 2553 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527119 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527122 2553 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 20:00:46.649508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527124 2553 flags.go:64] FLAG: --feature-gates="" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527130 2553 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527133 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527136 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527139 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527142 2553 flags.go:64] FLAG: --healthz-port="10248" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527145 2553 flags.go:64] FLAG: --help="false" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527148 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527151 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527154 2553 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527156 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527160 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527163 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527166 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527168 2553 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527171 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527174 2553 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527177 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527180 2553 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527183 2553 flags.go:64] FLAG: --kube-reserved="" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527185 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527188 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527191 2553 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527195 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 20:00:46.853602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527198 2553 flags.go:64] FLAG: --lock-file="" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527201 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527204 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527207 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527212 2553 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527216 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527219 2553 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527222 2553 flags.go:64] FLAG: --logging-format="text" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527224 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527228 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527230 2553 flags.go:64] FLAG: --manifest-url="" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527233 2553 flags.go:64] FLAG: --manifest-url-header="" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527237 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527240 2553 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527244 2553 flags.go:64] FLAG: --max-pods="110" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527247 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527250 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527252 2553 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527255 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527258 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527261 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527263 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527271 2553 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527274 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527277 2553 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 20:00:46.854660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527280 2553 flags.go:64] FLAG: --pod-cidr="" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527282 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527288 2553 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527291 2553 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527294 2553 flags.go:64] FLAG: --pods-per-core="0" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527296 2553 flags.go:64] FLAG: --port="10250" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527299 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527303 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01de6aec4ab4434af" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527307 2553 flags.go:64] FLAG: --qos-reserved="" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527310 2553 flags.go:64] FLAG: --read-only-port="10255" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527313 2553 flags.go:64] FLAG: --register-node="true" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527316 2553 flags.go:64] FLAG: --register-with-taints="" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527321 2553 flags.go:64] FLAG: --registry-burst="10" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527324 2553 flags.go:64] FLAG: --registry-qps="5" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527326 2553 flags.go:64] FLAG: --reserved-cpus="" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527329 2553 flags.go:64] FLAG: --reserved-memory="" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527332 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527335 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527338 2553 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527341 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527344 2553 flags.go:64] FLAG: --runonce="false" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527346 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527349 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527352 2553 flags.go:64] FLAG: --seccomp-default="false" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527355 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 20:00:46.864958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527358 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527361 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527364 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527366 2553 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527369 2553 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527372 2553 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527375 2553 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527377 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527380 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527383 2553 flags.go:64] FLAG: --system-cgroups="" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527386 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527391 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527393 2553 flags.go:64] FLAG: --tls-cert-file="" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527396 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527402 2553 flags.go:64] FLAG: --tls-min-version="" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527405 2553 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527407 2553 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527411 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527413 2553 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527418 2553 flags.go:64] FLAG: --v="2" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527421 2553 flags.go:64] FLAG: --version="false" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527425 2553 flags.go:64] FLAG: --vmodule="" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527429 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.527432 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527548 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 20:00:46.867751 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527551 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527555 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527558 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527561 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527564 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527567 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527569 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527572 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527575 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527578 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527580 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527583 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527585 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527588 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527590 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527593 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527596 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527598 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527600 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 20:00:46.870358 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527603 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527605 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527609 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527612 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527615 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527617 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527620 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527624 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527626 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527629 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527631 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527635 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527647 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527650 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527653 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527655 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527658 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527661 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527664 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527666 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 20:00:46.872453 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527668 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527671 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527674 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527676 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527679 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527682 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527684 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527687 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527689 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527693 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527696 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527699 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527702 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527705 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527709 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527712 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527715 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527718 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527720 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527724 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 20:00:46.873266 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527727 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527730 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527732 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527735 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527738 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527740 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527744 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527747 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527750 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527753 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527756 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527758 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527761 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527764 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527767 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527770 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527772 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527775 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527777 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 20:00:46.875518 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527780 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527782 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527785 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527787 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527789 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527792 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527794 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527799 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527801 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527803 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527806 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527809 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527812 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527815 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527818 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527820 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527823 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527825 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527828 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 20:00:46.877237 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.527830 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.528576 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.536409 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.536431 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.536447 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.536453 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536515 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536521 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536525 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536528 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536531 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536535 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536537 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536540 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 20:00:46.878403 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536543 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536545 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536548 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536551 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536553 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536556 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536558 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536561 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536564 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536566 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536569 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536571 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536574 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536576 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536579 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536581 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536584 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536586 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536589 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 20:00:46.878923 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536593 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536597 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536609 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536612 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536616 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536619 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536621 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536624 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536627 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536630 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536633 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536636 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536639 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536642 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536645 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536648 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536651 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536655 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536658 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 20:00:46.879546 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536661 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536664 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536667 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536670 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536673 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536675 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536678 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536680 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536683 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536685 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536688 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536690 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536693 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536696 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536698 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536702 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536710 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536713 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536716 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536718 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 20:00:46.880162 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536721 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536723 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536726 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536728 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536731 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536733 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536736 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536738 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536741 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536743 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536748 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536752 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536755 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536758 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536761 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536763 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536766 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536769 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536772 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536775 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 20:00:46.880771 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536778 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536780 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536783 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536786 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536789 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536792 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536795 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536801 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536804 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536812 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536815 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536817 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536820 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.536825 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536970 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 20:00:46.881385 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536975 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536978 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536981 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536984 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536986 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536989 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536991 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536996 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.536999 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537001 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537004 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537006 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537009 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537012 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537014 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537030 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537034 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537038 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537041 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537044 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 20:00:46.881994 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537046 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537049 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537052 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537055 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537058 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537062 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537064 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537073 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537076 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537079 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537081 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537084 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537086 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537089 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537092 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537095 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537098 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537100 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537103 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537107 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 20:00:46.882604 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537109 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537112 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537114 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537117 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537120 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537122 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537125 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537128 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537130 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537133 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537150 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537154 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537157 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537160 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537163 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537166 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537169 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537173 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537175 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537178 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 20:00:46.883454 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537188 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537191 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537194 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537196 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537199 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537202 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537204 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537207 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537210 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537212 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537215 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537219 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537221 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537224 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537226 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537228 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537231 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537234 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537236 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 20:00:46.884099 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537238 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537241 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537243 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537246 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537248 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537251 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537253 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537256 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537258 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537261 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537265 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537268 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537270 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537273 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537282 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537286 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537289 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537291 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 20:00:46.884655 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:46.537294 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.537298 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.537419 2553 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.539954 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.541889 2553 server.go:1019] "Starting client certificate rotation" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.541988 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.542046 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.544729 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.547255 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.561035 2553 log.go:25] "Validated CRI v1 runtime API" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.563415 2553 log.go:25] "Validated CRI v1 image API" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.564183 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 20:00:46.885282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.566826 2553 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 865b54c5-af20-40f8-8e7c-75406c2aaeb3:/dev/nvme0n1p3 f2b65300-916f-4567-b5d3-9b8afe741ae9:/dev/nvme0n1p4] Jul 07 20:00:46.885671 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.566850 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 20:00:46.886171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.573007 2553 manager.go:216] Machine: {Timestamp:2026-07-07 20:00:46.571344975 +0000 UTC m=+0.376990681 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104571 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d6590c73e85f3ff022f429e086e9f SystemUUID:ec2d6590-c73e-85f3-ff02-2f429e086e9f BootID:1246dd5c-d540-4e61-b2ff-696f67407330 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:d7:3e:33:eb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:d7:3e:33:eb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:df:be:99:16:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 20:00:46.886171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.573118 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 20:00:46.886171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.573198 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 20:00:46.886171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.576462 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 20:00:46.886171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.576534 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 20:00:46.886171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.576563 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-86.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 20:00:46.886171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.578116 2553 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.578126 2553 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.578138 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.578150 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.580052 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.584730 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.584855 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.584885 2553 kubelet.go:486] "Attempting to sync node with API server" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.584895 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.584909 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.584921 2553 kubelet.go:398] "Adding apiserver pod source" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.584931 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.585295 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.585315 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.587671 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.591099 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.591126 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592491 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592504 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592511 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592526 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592535 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592543 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592553 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592559 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592565 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592571 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592583 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.592588 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.593535 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.593545 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.597082 2553 server.go:1268] "Started kubelet" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.597202 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-86.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.597282 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.597359 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.597379 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.598745 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.602723 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.604527 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.605181 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 20:00:46.886465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.605722 2553 server.go:310] "Adding debug handlers to kubelet server" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.607743 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-86.ec2.internal.18c01a5eba823526 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-86.ec2.internal,UID:ip-10-0-133-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-86.ec2.internal,},FirstTimestamp:2026-07-07 20:00:46.597059878 +0000 UTC m=+0.402705578,LastTimestamp:2026-07-07 20:00:46.597059878 +0000 UTC m=+0.402705578,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-86.ec2.internal,}" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.613091 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.613109 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.613194 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.613886 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.613889 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.613906 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.613908 2553 factory.go:55] Registering systemd factory Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.613924 2553 factory.go:223] Registration of the systemd container factory successfully Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614056 2553 reconstruct.go:97] "Volume reconstruction finished" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614064 2553 reconciler.go:29] "Reconciler: start to sync state" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614126 2553 factory.go:153] Registering CRI-O factory Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614139 2553 factory.go:223] Registration of the crio container factory successfully Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614191 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614214 2553 factory.go:103] Registering Raw factory Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614228 2553 manager.go:1196] Started watching for new ooms in manager Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.614568 2553 manager.go:349] Starting recovery of all containers Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.617500 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.617921 2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.623752 2553 manager.go:354] Recovery completed Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.628376 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.630378 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientMemory" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.630404 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.630425 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientPID" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.630847 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.630854 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.630868 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.630913 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.631544 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientMemory" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.631567 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 20:00:46.994674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.631576 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientPID" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.632799 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-86.ec2.internal.18c01a5ebc7ecb74 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-86.ec2.internal,UID:ip-10-0-133-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-86.ec2.internal,},FirstTimestamp:2026-07-07 20:00:46.630390644 +0000 UTC m=+0.436036345,LastTimestamp:2026-07-07 20:00:46.630390644 +0000 UTC m=+0.436036345,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-86.ec2.internal,}" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.633045 2553 policy_none.go:49] "None policy: Start" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.633060 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.633074 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.635813 2553 policy_none.go:47] "Start" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.644371 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-86.ec2.internal.18c01a5ebc7f3d6d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-86.ec2.internal,UID:ip-10-0-133-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-86.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-86.ec2.internal,},FirstTimestamp:2026-07-07 20:00:46.630419821 +0000 UTC m=+0.436065520,LastTimestamp:2026-07-07 20:00:46.630419821 +0000 UTC m=+0.436065520,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-86.ec2.internal,}" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.657162 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-86.ec2.internal.18c01a5ebc7f6495 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-86.ec2.internal,UID:ip-10-0-133-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-86.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-86.ec2.internal,},FirstTimestamp:2026-07-07 20:00:46.630429845 +0000 UTC m=+0.436075543,LastTimestamp:2026-07-07 20:00:46.630429845 +0000 UTC m=+0.436075543,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-86.ec2.internal,}" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.666068 2553 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-86.ec2.internal.18c01a5ebc7ecb74\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-86.ec2.internal.18c01a5ebc7ecb74 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-86.ec2.internal,UID:ip-10-0-133-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-86.ec2.internal,},FirstTimestamp:2026-07-07 20:00:46.630390644 +0000 UTC m=+0.436036345,LastTimestamp:2026-07-07 20:00:46.631554311 +0000 UTC m=+0.437200009,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-86.ec2.internal,}" Jul 07 20:00:46.995702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.666456 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wrnhh" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.674553 2553 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-86.ec2.internal.18c01a5ebc7f3d6d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-86.ec2.internal.18c01a5ebc7f3d6d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-86.ec2.internal,UID:ip-10-0-133-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-86.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-86.ec2.internal,},FirstTimestamp:2026-07-07 20:00:46.630419821 +0000 UTC m=+0.436065520,LastTimestamp:2026-07-07 20:00:46.631571007 +0000 UTC m=+0.437216705,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-86.ec2.internal,}" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.676728 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wrnhh" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.713636 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.713718 2553 manager.go:337] "Starting Device Plugin manager" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.713748 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.713757 2553 server.go:87] "Starting device plugin registration server" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.714048 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.714063 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.714215 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.714305 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.714316 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.715071 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.715124 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.724424 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.725636 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.725671 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.725690 2553 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.725729 2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.728414 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.814817 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.815805 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientMemory" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.815834 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.815844 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientPID" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.815872 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.825761 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal"] Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.825815 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.827107 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientMemory" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.827131 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.827144 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientPID" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.828105 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996008 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.828128 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-86.ec2.internal\": node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.829906 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830003 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830050 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830578 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientMemory" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830583 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientMemory" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830599 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830605 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830609 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientPID" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.830620 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientPID" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.832794 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.832818 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.833517 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientMemory" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.833544 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.833565 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeHasSufficientPID" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.882941 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.898645 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-86.ec2.internal\" not found" node="ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.900780 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-86.ec2.internal\" not found" node="ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.915155 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42e328737f94c0a05dc46b508bb8438-config\") pod \"kube-apiserver-proxy-ip-10-0-133-86.ec2.internal\" (UID: \"e42e328737f94c0a05dc46b508bb8438\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.915176 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1a88a977f1e251bcb667d906b23f249-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal\" (UID: \"a1a88a977f1e251bcb667d906b23f249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:46.915192 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1a88a977f1e251bcb667d906b23f249-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal\" (UID: \"a1a88a977f1e251bcb667d906b23f249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:46.996905 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:46.984341 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:47.098064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.015556 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1a88a977f1e251bcb667d906b23f249-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal\" (UID: \"a1a88a977f1e251bcb667d906b23f249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.098064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.015578 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42e328737f94c0a05dc46b508bb8438-config\") pod \"kube-apiserver-proxy-ip-10-0-133-86.ec2.internal\" (UID: \"e42e328737f94c0a05dc46b508bb8438\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.098064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.015595 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1a88a977f1e251bcb667d906b23f249-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal\" (UID: \"a1a88a977f1e251bcb667d906b23f249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.098064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.015634 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1a88a977f1e251bcb667d906b23f249-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal\" (UID: \"a1a88a977f1e251bcb667d906b23f249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.098064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.015675 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1a88a977f1e251bcb667d906b23f249-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal\" (UID: \"a1a88a977f1e251bcb667d906b23f249\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.098064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.015688 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42e328737f94c0a05dc46b508bb8438-config\") pod \"kube-apiserver-proxy-ip-10-0-133-86.ec2.internal\" (UID: \"e42e328737f94c0a05dc46b508bb8438\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.098064 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:47.084804 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:47.220673 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:47.185353 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:47.220673 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.202506 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.220673 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:47.204619 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:47.314251 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:47.285787 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:47.416380 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:47.386815 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:48.825348 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:48.784248 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:48.932407 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:48.837718 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 20:00:48.932407 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:48.837857 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 20:00:48.932407 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:48.885150 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:49.044054 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:48.976436 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 19:55:47 +0000 UTC" deadline="2028-03-28 01:39:01.765970488 +0000 UTC" Jul 07 20:00:49.044054 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:48.976476 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15101h38m12.789496955s" Jul 07 20:00:49.044054 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:48.985569 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-86.ec2.internal\" not found" Jul 07 20:00:49.044054 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.017138 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 20:00:49.060076 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.059075 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 20:00:49.158173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.111177 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" Jul 07 20:00:49.158173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.123638 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 20:00:49.158173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.124534 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" Jul 07 20:00:49.158173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.139001 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 20:00:49.664787 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.600242 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 20:00:49.664787 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:49.606239 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a88a977f1e251bcb667d906b23f249.slice/crio-49c95fe41c8f26f81020a2c5baf2b67d99491714867faa48236bc512cd1abf13 WatchSource:0}: Error finding container 49c95fe41c8f26f81020a2c5baf2b67d99491714867faa48236bc512cd1abf13: Status 404 returned error can't find the container with id 49c95fe41c8f26f81020a2c5baf2b67d99491714867faa48236bc512cd1abf13 Jul 07 20:00:49.664787 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.610257 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 20:00:49.701069 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.701015 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 20:00:49.722148 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:49.722128 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42e328737f94c0a05dc46b508bb8438.slice/crio-a3d9777e0a853fcbabaee1af2426c861fd89f4bba05fff0ce974f8e2af655ec4 WatchSource:0}: Error finding container a3d9777e0a853fcbabaee1af2426c861fd89f4bba05fff0ce974f8e2af655ec4: Status 404 returned error can't find the container with id a3d9777e0a853fcbabaee1af2426c861fd89f4bba05fff0ce974f8e2af655ec4 Jul 07 20:00:49.883102 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.883072 2553 apiserver.go:52] "Watching apiserver" Jul 07 20:00:49.892194 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.892158 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 20:00:49.892498 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.892478 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dsxqc","kube-system/konnectivity-agent-4d29q","kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542","openshift-cluster-node-tuning-operator/tuned-7fwgt","openshift-dns/node-resolver-t7q6z","openshift-image-registry/node-ca-lqwnx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal","openshift-multus/multus-additional-cni-plugins-vjj97","openshift-multus/multus-z27qz","openshift-multus/network-metrics-daemon-dt99m","openshift-network-diagnostics/network-check-target-47mc5","openshift-network-operator/iptables-alerter-grkwb","openshift-ovn-kubernetes/ovnkube-node-tk7bx"] Jul 07 20:00:49.895149 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.895136 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:49.897143 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.897120 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:49.898028 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.898007 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2tgj9\"" Jul 07 20:00:49.898121 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.898102 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 20:00:49.898201 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.898128 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 20:00:49.899306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.899282 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.899642 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.899593 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:00:49.899642 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.899636 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 20:00:49.900240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.899805 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 20:00:49.900240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.899952 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 20:00:49.900240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.900037 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4gdht\"" Jul 07 20:00:49.902041 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.901468 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:49.902041 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.901897 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 20:00:49.902552 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.902341 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 20:00:49.902552 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.902341 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6v9cd\"" Jul 07 20:00:49.902552 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.902457 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 20:00:49.903516 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.903499 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 20:00:49.903589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.903554 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 20:00:49.903589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.903567 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 20:00:49.903789 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.903773 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.904190 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.904139 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 20:00:49.904289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.904226 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 20:00:49.904289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.904274 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 20:00:49.904568 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.904555 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-blhsp\"" Jul 07 20:00:49.906032 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.906012 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:49.906104 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.906041 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:00:49.906394 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.906376 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rlrgc\"" Jul 07 20:00:49.906518 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.906501 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 20:00:49.908577 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.908560 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:49.908665 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.908624 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 20:00:49.909008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.908975 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 20:00:49.909629 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.909610 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7fqh6\"" Jul 07 20:00:49.911077 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.911061 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 20:00:49.911150 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.911068 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:49.911218 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:49.911197 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:00:49.911537 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.911522 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 20:00:49.911641 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.911627 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 20:00:49.911674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.911633 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 20:00:49.911720 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.911696 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 20:00:49.912043 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.912024 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 20:00:49.912145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.912131 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k7lct\"" Jul 07 20:00:49.913386 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.913373 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:49.916400 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.916055 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 20:00:49.916400 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.916145 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z27qz" Jul 07 20:00:49.916400 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.916064 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hl2ps\"" Jul 07 20:00:49.916545 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.916520 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 20:00:49.918984 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.918962 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 20:00:49.919492 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.919475 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-prgz7\"" Jul 07 20:00:49.919830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.919813 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:49.919900 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:49.919872 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:00:49.922229 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.922216 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:49.922319 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:49.922301 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:00:49.924631 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.924604 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 20:00:49.954346 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.954305 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pppz8" Jul 07 20:00:49.964669 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.964651 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pppz8" Jul 07 20:00:49.998979 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.998957 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-os-release\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:49.999075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.998980 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fde5b076-8694-4377-b36d-e5b7656f874f-agent-certs\") pod \"konnectivity-agent-4d29q\" (UID: \"fde5b076-8694-4377-b36d-e5b7656f874f\") " pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:49.999075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999008 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-ovn\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999023 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-device-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:49.999075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999038 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj6dz\" (UniqueName: \"kubernetes.io/projected/f411591f-909b-46e9-bdd6-812010f8f517-kube-api-access-dj6dz\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:49.999075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999052 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6405eede-1bde-4bf5-b476-ce244e0029eb-host\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:49.999075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999066 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cnibin\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:49.999246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999090 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:49.999246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999104 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-run-ovn-kubernetes\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999155 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-log-socket\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999184 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-sys-fs\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:49.999246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999204 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysconfig\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.999246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999227 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999248 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-slash\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999261 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fde5b076-8694-4377-b36d-e5b7656f874f-konnectivity-ca\") pod \"konnectivity-agent-4d29q\" (UID: \"fde5b076-8694-4377-b36d-e5b7656f874f\") " pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999286 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-run-netns\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999326 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysctl-d\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999346 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysctl-conf\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999360 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-var-lib-kubelet\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999374 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-ovnkube-script-lib\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999394 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8qzn\" (UniqueName: \"kubernetes.io/projected/8297ef12-5367-4138-8cc1-1beba8252179-kube-api-access-v8qzn\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999409 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e356e3c-7634-4b62-b81d-46cb0a409d97-dbus\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999422 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c42d5506-06d0-4db9-9172-933faebfaee7-tmp\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999449 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhf6\" (UniqueName: \"kubernetes.io/projected/2a31e2bb-5d62-4b23-a428-9943115b4cd3-kube-api-access-fdhf6\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999491 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e356e3c-7634-4b62-b81d-46cb0a409d97-kubelet-config\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999506 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2v8d\" (UniqueName: \"kubernetes.io/projected/6405eede-1bde-4bf5-b476-ce244e0029eb-kube-api-access-s2v8d\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999529 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-system-cni-dir\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999556 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-var-lib-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999571 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-ovnkube-config\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999594 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-etc-selinux\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999613 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-systemd\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.999657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999644 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e558b3a-f880-4d8e-84c4-cf3268697b76-iptables-alerter-script\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999665 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999680 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-node-log\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999720 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8297ef12-5367-4138-8cc1-1beba8252179-ovn-node-metrics-cert\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999749 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a31e2bb-5d62-4b23-a428-9943115b4cd3-hosts-file\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999770 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-kubelet\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999786 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-systemd\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999810 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-cni-bin\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999851 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999865 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a31e2bb-5d62-4b23-a428-9943115b4cd3-tmp-dir\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999881 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-registration-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999899 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-kubernetes\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:49.999922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999914 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-sys\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999928 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbh2l\" (UniqueName: \"kubernetes.io/projected/c42d5506-06d0-4db9-9172-933faebfaee7-kube-api-access-qbh2l\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999946 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999970 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsprl\" (UniqueName: \"kubernetes.io/projected/2e558b3a-f880-4d8e-84c4-cf3268697b76-kube-api-access-fsprl\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:49.999985 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-modprobe-d\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000012 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-lib-modules\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000033 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c42d5506-06d0-4db9-9172-933faebfaee7-etc-tuned\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000055 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-systemd-units\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000072 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-cni-netd\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000088 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000105 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-socket-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000146 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000171 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkn26\" (UniqueName: \"kubernetes.io/projected/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-kube-api-access-tkn26\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.000291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000196 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e558b3a-f880-4d8e-84c4-cf3268697b76-host-slash\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.000698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000225 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-etc-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.000698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000244 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.000698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000282 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6405eede-1bde-4bf5-b476-ce244e0029eb-serviceca\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.000698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000304 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-run\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.000698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000322 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-host\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.000698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.000342 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-env-overrides\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.012257 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.012238 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 20:00:50.026115 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.026081 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" event={"ID":"e42e328737f94c0a05dc46b508bb8438","Type":"ContainerStarted","Data":"a3d9777e0a853fcbabaee1af2426c861fd89f4bba05fff0ce974f8e2af655ec4"} Jul 07 20:00:50.027117 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.027100 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" event={"ID":"a1a88a977f1e251bcb667d906b23f249","Type":"ContainerStarted","Data":"49c95fe41c8f26f81020a2c5baf2b67d99491714867faa48236bc512cd1abf13"} Jul 07 20:00:50.100629 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100600 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-etc-kubernetes\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.100765 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100643 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2v8d\" (UniqueName: \"kubernetes.io/projected/6405eede-1bde-4bf5-b476-ce244e0029eb-kube-api-access-s2v8d\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.100765 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100687 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-system-cni-dir\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.100765 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100720 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-var-lib-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.100765 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100745 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-ovnkube-config\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.100959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100768 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-etc-selinux\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.100959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100793 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-systemd\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.100959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100791 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-system-cni-dir\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.100959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100803 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-var-lib-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.100959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100858 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-systemd\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.100959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100900 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e558b3a-f880-4d8e-84c4-cf3268697b76-iptables-alerter-script\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.100959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100948 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.100978 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-node-log\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101030 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-etc-selinux\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.101219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101044 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-node-log\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101121 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101163 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8297ef12-5367-4138-8cc1-1beba8252179-ovn-node-metrics-cert\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101197 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101226 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a31e2bb-5d62-4b23-a428-9943115b4cd3-hosts-file\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101251 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-kubelet\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101272 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-systemd\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101294 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-cni-bin\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101319 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101318 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-kubelet\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101348 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101353 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-cni-bin\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101321 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-systemd\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101298 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a31e2bb-5d62-4b23-a428-9943115b4cd3-hosts-file\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101377 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a31e2bb-5d62-4b23-a428-9943115b4cd3-tmp-dir\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101399 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-registration-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101421 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-cni-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.101425 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:50.101481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101443 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-k8s-cni-cncf-io\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.102036 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101471 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-kubernetes\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102036 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101497 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-sys\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102036 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101521 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbh2l\" (UniqueName: \"kubernetes.io/projected/c42d5506-06d0-4db9-9172-933faebfaee7-kube-api-access-qbh2l\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102036 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101930 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 20:00:50.102036 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101941 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a31e2bb-5d62-4b23-a428-9943115b4cd3-tmp-dir\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:50.102036 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101951 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.102262 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.102067 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret podName:6e356e3c-7634-4b62-b81d-46cb0a409d97 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:50.602019624 +0000 UTC m=+3.110692184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret") pod "global-pull-secret-syncer-dsxqc" (UID: "6e356e3c-7634-4b62-b81d-46cb0a409d97") : object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:50.102262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.101695 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-ovnkube-config\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.102262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102107 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsprl\" (UniqueName: \"kubernetes.io/projected/2e558b3a-f880-4d8e-84c4-cf3268697b76-kube-api-access-fsprl\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.102262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102142 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-kubernetes\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102186 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7shvt\" (UniqueName: \"kubernetes.io/projected/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-kube-api-access-7shvt\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:50.102262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102224 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-cnibin\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.102459 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102297 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-modprobe-d\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102459 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102317 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-registration-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.102459 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102421 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-sys\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102572 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102532 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-modprobe-d\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102598 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-lib-modules\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102730 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c42d5506-06d0-4db9-9172-933faebfaee7-etc-tuned\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102814 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102788 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-lib-modules\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.102865 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102855 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-systemd-units\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.102920 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102896 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-systemd-units\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.102983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.102967 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-cni-netd\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103038 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103062 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103135 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-socket-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103159 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103166 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-socket-dir-parent\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103214 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-cni-netd\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103221 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103257 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e558b3a-f880-4d8e-84c4-cf3268697b76-iptables-alerter-script\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103289 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkn26\" (UniqueName: \"kubernetes.io/projected/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-kube-api-access-tkn26\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.103385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103324 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e558b3a-f880-4d8e-84c4-cf3268697b76-host-slash\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103451 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-socket-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103550 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-etc-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103595 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103626 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-os-release\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103655 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-cni-bin\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103690 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6405eede-1bde-4bf5-b476-ce244e0029eb-serviceca\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103705 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103722 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-run\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103753 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-host\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103780 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.103861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103778 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-env-overrides\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.104344 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103880 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e558b3a-f880-4d8e-84c4-cf3268697b76-host-slash\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.104344 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.103937 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-etc-openvswitch\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104507 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-env-overrides\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104514 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6405eede-1bde-4bf5-b476-ce244e0029eb-serviceca\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104548 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-cni-multus\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104634 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-host\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104629 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-run\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-daemon-config\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104882 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-os-release\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104908 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fde5b076-8694-4377-b36d-e5b7656f874f-agent-certs\") pod \"konnectivity-agent-4d29q\" (UID: \"fde5b076-8694-4377-b36d-e5b7656f874f\") " pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104938 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-ovn\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.104972 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-device-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105020 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj6dz\" (UniqueName: \"kubernetes.io/projected/f411591f-909b-46e9-bdd6-812010f8f517-kube-api-access-dj6dz\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105048 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-kubelet\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105074 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6405eede-1bde-4bf5-b476-ce244e0029eb-host\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.105774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105104 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cnibin\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105132 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105165 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-run-ovn-kubernetes\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105199 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-system-cni-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105224 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9817fa91-e60e-4cb5-b49d-40af3b1841a3-cni-binary-copy\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105253 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-conf-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105308 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-log-socket\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-sys-fs\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105364 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysconfig\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105391 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105420 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-slash\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105452 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-multus-certs\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105481 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fde5b076-8694-4377-b36d-e5b7656f874f-konnectivity-ca\") pod \"konnectivity-agent-4d29q\" (UID: \"fde5b076-8694-4377-b36d-e5b7656f874f\") " pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105514 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-run-netns\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.106574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105590 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-device-dir\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105612 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8hmv\" (UniqueName: \"kubernetes.io/projected/9817fa91-e60e-4cb5-b49d-40af3b1841a3-kube-api-access-j8hmv\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105669 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-log-socket\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105673 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysconfig\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105707 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysctl-d\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105660 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105785 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cnibin\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105887 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-run-netns\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105931 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysctl-conf\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105956 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-run-ovn-kubernetes\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.105962 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-var-lib-kubelet\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106038 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-var-lib-kubelet\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106068 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f411591f-909b-46e9-bdd6-812010f8f517-sys-fs\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106142 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysctl-conf\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106175 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-run-ovn\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.107239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106239 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6405eede-1bde-4bf5-b476-ce244e0029eb-host\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106264 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c42d5506-06d0-4db9-9172-933faebfaee7-etc-sysctl-d\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106287 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8297ef12-5367-4138-8cc1-1beba8252179-host-slash\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106421 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-ovnkube-script-lib\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106478 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8qzn\" (UniqueName: \"kubernetes.io/projected/8297ef12-5367-4138-8cc1-1beba8252179-kube-api-access-v8qzn\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106511 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e356e3c-7634-4b62-b81d-46cb0a409d97-dbus\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106561 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-os-release\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106580 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fde5b076-8694-4377-b36d-e5b7656f874f-konnectivity-ca\") pod \"konnectivity-agent-4d29q\" (UID: \"fde5b076-8694-4377-b36d-e5b7656f874f\") " pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106627 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-hostroot\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106670 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e356e3c-7634-4b62-b81d-46cb0a409d97-dbus\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106687 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c42d5506-06d0-4db9-9172-933faebfaee7-tmp\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106746 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhf6\" (UniqueName: \"kubernetes.io/projected/2a31e2bb-5d62-4b23-a428-9943115b4cd3-kube-api-access-fdhf6\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106818 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106839 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e356e3c-7634-4b62-b81d-46cb0a409d97-kubelet-config\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.106954 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-netns\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.107839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.107288 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8297ef12-5367-4138-8cc1-1beba8252179-ovn-node-metrics-cert\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.108367 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.107313 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8297ef12-5367-4138-8cc1-1beba8252179-ovnkube-script-lib\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.108367 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.107377 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e356e3c-7634-4b62-b81d-46cb0a409d97-kubelet-config\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:50.108367 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.108244 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c42d5506-06d0-4db9-9172-933faebfaee7-etc-tuned\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.108528 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.108512 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fde5b076-8694-4377-b36d-e5b7656f874f-agent-certs\") pod \"konnectivity-agent-4d29q\" (UID: \"fde5b076-8694-4377-b36d-e5b7656f874f\") " pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:50.109268 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.109250 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c42d5506-06d0-4db9-9172-933faebfaee7-tmp\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.110584 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.110565 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2v8d\" (UniqueName: \"kubernetes.io/projected/6405eede-1bde-4bf5-b476-ce244e0029eb-kube-api-access-s2v8d\") pod \"node-ca-lqwnx\" (UID: \"6405eede-1bde-4bf5-b476-ce244e0029eb\") " pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.113344 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.113328 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsprl\" (UniqueName: \"kubernetes.io/projected/2e558b3a-f880-4d8e-84c4-cf3268697b76-kube-api-access-fsprl\") pod \"iptables-alerter-grkwb\" (UID: \"2e558b3a-f880-4d8e-84c4-cf3268697b76\") " pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.117608 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.117589 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkn26\" (UniqueName: \"kubernetes.io/projected/bd18ce10-f49a-4a99-96c1-cefbccfb6c80-kube-api-access-tkn26\") pod \"multus-additional-cni-plugins-vjj97\" (UID: \"bd18ce10-f49a-4a99-96c1-cefbccfb6c80\") " pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.119648 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.119632 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbh2l\" (UniqueName: \"kubernetes.io/projected/c42d5506-06d0-4db9-9172-933faebfaee7-kube-api-access-qbh2l\") pod \"tuned-7fwgt\" (UID: \"c42d5506-06d0-4db9-9172-933faebfaee7\") " pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.121654 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.121615 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj6dz\" (UniqueName: \"kubernetes.io/projected/f411591f-909b-46e9-bdd6-812010f8f517-kube-api-access-dj6dz\") pod \"aws-ebs-csi-driver-node-xs542\" (UID: \"f411591f-909b-46e9-bdd6-812010f8f517\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.121848 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.121830 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8qzn\" (UniqueName: \"kubernetes.io/projected/8297ef12-5367-4138-8cc1-1beba8252179-kube-api-access-v8qzn\") pod \"ovnkube-node-tk7bx\" (UID: \"8297ef12-5367-4138-8cc1-1beba8252179\") " pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.121902 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.121869 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhf6\" (UniqueName: \"kubernetes.io/projected/2a31e2bb-5d62-4b23-a428-9943115b4cd3-kube-api-access-fdhf6\") pod \"node-resolver-t7q6z\" (UID: \"2a31e2bb-5d62-4b23-a428-9943115b4cd3\") " pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:50.205165 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.205099 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lqwnx" Jul 07 20:00:50.207890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207805 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-kubelet\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.207890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207843 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-system-cni-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.207890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207867 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9817fa91-e60e-4cb5-b49d-40af3b1841a3-cni-binary-copy\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207891 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-conf-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207922 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-multus-certs\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207947 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8hmv\" (UniqueName: \"kubernetes.io/projected/9817fa91-e60e-4cb5-b49d-40af3b1841a3-kube-api-access-j8hmv\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207948 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-kubelet\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207976 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-hostroot\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208036 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-netns\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208039 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-conf-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208044 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-multus-certs\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208060 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-etc-kubernetes\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208088 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.207974 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-system-cni-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208102 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208107 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-netns\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208142 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208148 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-hostroot\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208179 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-cni-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208207 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-etc-kubernetes\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208220 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-k8s-cni-cncf-io\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208253 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7shvt\" (UniqueName: \"kubernetes.io/projected/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-kube-api-access-7shvt\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208276 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-cnibin\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.208288 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208305 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-socket-dir-parent\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-os-release\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208366 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-cni-bin\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208393 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-cni-multus\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.208585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208418 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-daemon-config\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208451 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9817fa91-e60e-4cb5-b49d-40af3b1841a3-cni-binary-copy\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208493 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-run-k8s-cni-cncf-io\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208525 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-os-release\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.208546 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:00:50.708510725 +0000 UTC m=+3.217183282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208551 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-cni-bin\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208325 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-cni-dir\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208589 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-host-var-lib-cni-multus\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208627 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-cnibin\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208654 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-socket-dir-parent\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.209277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.208869 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9817fa91-e60e-4cb5-b49d-40af3b1841a3-multus-daemon-config\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.212289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.212267 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-grkwb" Jul 07 20:00:50.215007 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.214973 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6405eede_1bde_4bf5_b476_ce244e0029eb.slice/crio-14236a20bd6c2e9cf2a02465836ed72613e6615b8e0834fdb34268d9e2e2ef48 WatchSource:0}: Error finding container 14236a20bd6c2e9cf2a02465836ed72613e6615b8e0834fdb34268d9e2e2ef48: Status 404 returned error can't find the container with id 14236a20bd6c2e9cf2a02465836ed72613e6615b8e0834fdb34268d9e2e2ef48 Jul 07 20:00:50.220034 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.219633 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 20:00:50.220034 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.219653 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 20:00:50.220034 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.219666 2553 projected.go:196] Error preparing data for projected volume kube-api-access-4tdjg for pod openshift-network-diagnostics/network-check-target-47mc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:50.220034 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.219734 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg podName:86bcbc7a-039d-4f27-b439-6b6aad64f748 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:50.719717747 +0000 UTC m=+3.228390306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4tdjg" (UniqueName: "kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg") pod "network-check-target-47mc5" (UID: "86bcbc7a-039d-4f27-b439-6b6aad64f748") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:50.220741 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.220723 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:00:50.221174 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.221157 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8hmv\" (UniqueName: \"kubernetes.io/projected/9817fa91-e60e-4cb5-b49d-40af3b1841a3-kube-api-access-j8hmv\") pod \"multus-z27qz\" (UID: \"9817fa91-e60e-4cb5-b49d-40af3b1841a3\") " pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.221465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.221446 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7shvt\" (UniqueName: \"kubernetes.io/projected/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-kube-api-access-7shvt\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:50.222588 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.222568 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e558b3a_f880_4d8e_84c4_cf3268697b76.slice/crio-01c4e300072e6913d4ef6177080f9fec979196a890b7ec90ac8e0d1adff3c026 WatchSource:0}: Error finding container 01c4e300072e6913d4ef6177080f9fec979196a890b7ec90ac8e0d1adff3c026: Status 404 returned error can't find the container with id 01c4e300072e6913d4ef6177080f9fec979196a890b7ec90ac8e0d1adff3c026 Jul 07 20:00:50.224485 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.224319 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" Jul 07 20:00:50.229926 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.229902 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" Jul 07 20:00:50.231636 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.231613 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8297ef12_5367_4138_8cc1_1beba8252179.slice/crio-0cb8a12ff92512741339a728b30dbc3375bdf1d1430234348c00eb86bc379629 WatchSource:0}: Error finding container 0cb8a12ff92512741339a728b30dbc3375bdf1d1430234348c00eb86bc379629: Status 404 returned error can't find the container with id 0cb8a12ff92512741339a728b30dbc3375bdf1d1430234348c00eb86bc379629 Jul 07 20:00:50.234514 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.234496 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t7q6z" Jul 07 20:00:50.235601 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.235549 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf411591f_909b_46e9_bdd6_812010f8f517.slice/crio-79febbe3f2067024ce1ecdf897ff1a9efa4d39ee1f34a5161a76c8a91c5cc851 WatchSource:0}: Error finding container 79febbe3f2067024ce1ecdf897ff1a9efa4d39ee1f34a5161a76c8a91c5cc851: Status 404 returned error can't find the container with id 79febbe3f2067024ce1ecdf897ff1a9efa4d39ee1f34a5161a76c8a91c5cc851 Jul 07 20:00:50.240559 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.240543 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjj97" Jul 07 20:00:50.240861 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.240839 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42d5506_06d0_4db9_9172_933faebfaee7.slice/crio-198c4dba7caa47450a102c5ecf97e7009977093620df487ea30fee0a46bf9617 WatchSource:0}: Error finding container 198c4dba7caa47450a102c5ecf97e7009977093620df487ea30fee0a46bf9617: Status 404 returned error can't find the container with id 198c4dba7caa47450a102c5ecf97e7009977093620df487ea30fee0a46bf9617 Jul 07 20:00:50.245674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.245518 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:00:50.249617 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.249599 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z27qz" Jul 07 20:00:50.249905 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.249879 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a31e2bb_5d62_4b23_a428_9943115b4cd3.slice/crio-10a90aea5cae0940598abb9e1eb664f392caed3f647d6b1bb356b16cdcbc93eb WatchSource:0}: Error finding container 10a90aea5cae0940598abb9e1eb664f392caed3f647d6b1bb356b16cdcbc93eb: Status 404 returned error can't find the container with id 10a90aea5cae0940598abb9e1eb664f392caed3f647d6b1bb356b16cdcbc93eb Jul 07 20:00:50.254183 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.254162 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd18ce10_f49a_4a99_96c1_cefbccfb6c80.slice/crio-a9f2b391dee97b3d8275912fe66d3d6032303e45bd2ae2e47750d73de7b1d26f WatchSource:0}: Error finding container a9f2b391dee97b3d8275912fe66d3d6032303e45bd2ae2e47750d73de7b1d26f: Status 404 returned error can't find the container with id a9f2b391dee97b3d8275912fe66d3d6032303e45bd2ae2e47750d73de7b1d26f Jul 07 20:00:50.257623 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.257592 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde5b076_8694_4377_b36d_e5b7656f874f.slice/crio-0f6e66f1ff73963c97083144afa1164d19e532a38a36eb2a35bdb3e127fb6fcc WatchSource:0}: Error finding container 0f6e66f1ff73963c97083144afa1164d19e532a38a36eb2a35bdb3e127fb6fcc: Status 404 returned error can't find the container with id 0f6e66f1ff73963c97083144afa1164d19e532a38a36eb2a35bdb3e127fb6fcc Jul 07 20:00:50.261599 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:00:50.261559 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9817fa91_e60e_4cb5_b49d_40af3b1841a3.slice/crio-1300bfc69538accbea619740070db346f115bdcb18ecc964756e8f5f2561859e WatchSource:0}: Error finding container 1300bfc69538accbea619740070db346f115bdcb18ecc964756e8f5f2561859e: Status 404 returned error can't find the container with id 1300bfc69538accbea619740070db346f115bdcb18ecc964756e8f5f2561859e Jul 07 20:00:50.611035 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.610943 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:50.611166 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.611104 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:50.611211 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.611169 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret podName:6e356e3c-7634-4b62-b81d-46cb0a409d97 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:51.611153367 +0000 UTC m=+4.119825920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret") pod "global-pull-secret-syncer-dsxqc" (UID: "6e356e3c-7634-4b62-b81d-46cb0a409d97") : object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:50.711718 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.711679 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:50.711902 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.711879 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:50.711978 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.711967 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:00:51.711945289 +0000 UTC m=+4.220617844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:50.813020 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.812975 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:50.813212 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.813183 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 20:00:50.813212 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.813211 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 20:00:50.813360 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.813225 2553 projected.go:196] Error preparing data for projected volume kube-api-access-4tdjg for pod openshift-network-diagnostics/network-check-target-47mc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:50.813360 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:50.813283 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg podName:86bcbc7a-039d-4f27-b439-6b6aad64f748 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:51.813264765 +0000 UTC m=+4.321937336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4tdjg" (UniqueName: "kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg") pod "network-check-target-47mc5" (UID: "86bcbc7a-039d-4f27-b439-6b6aad64f748") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:50.968329 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.968237 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:55:49 +0000 UTC" deadline="2028-03-13 06:35:36.168788935 +0000 UTC" Jul 07 20:00:50.968329 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:50.968278 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14746h34m45.200515365s" Jul 07 20:00:51.023466 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.023361 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:51.023632 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.023489 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:00:51.030192 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.030153 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z27qz" event={"ID":"9817fa91-e60e-4cb5-b49d-40af3b1841a3","Type":"ContainerStarted","Data":"1300bfc69538accbea619740070db346f115bdcb18ecc964756e8f5f2561859e"} Jul 07 20:00:51.031742 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.031711 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4d29q" event={"ID":"fde5b076-8694-4377-b36d-e5b7656f874f","Type":"ContainerStarted","Data":"0f6e66f1ff73963c97083144afa1164d19e532a38a36eb2a35bdb3e127fb6fcc"} Jul 07 20:00:51.033132 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.033099 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerStarted","Data":"a9f2b391dee97b3d8275912fe66d3d6032303e45bd2ae2e47750d73de7b1d26f"} Jul 07 20:00:51.034388 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.034363 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t7q6z" event={"ID":"2a31e2bb-5d62-4b23-a428-9943115b4cd3","Type":"ContainerStarted","Data":"10a90aea5cae0940598abb9e1eb664f392caed3f647d6b1bb356b16cdcbc93eb"} Jul 07 20:00:51.035623 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.035596 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" event={"ID":"c42d5506-06d0-4db9-9172-933faebfaee7","Type":"ContainerStarted","Data":"198c4dba7caa47450a102c5ecf97e7009977093620df487ea30fee0a46bf9617"} Jul 07 20:00:51.037205 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.037181 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" event={"ID":"f411591f-909b-46e9-bdd6-812010f8f517","Type":"ContainerStarted","Data":"79febbe3f2067024ce1ecdf897ff1a9efa4d39ee1f34a5161a76c8a91c5cc851"} Jul 07 20:00:51.038520 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.038497 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"0cb8a12ff92512741339a728b30dbc3375bdf1d1430234348c00eb86bc379629"} Jul 07 20:00:51.039863 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.039841 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-grkwb" event={"ID":"2e558b3a-f880-4d8e-84c4-cf3268697b76","Type":"ContainerStarted","Data":"01c4e300072e6913d4ef6177080f9fec979196a890b7ec90ac8e0d1adff3c026"} Jul 07 20:00:51.041115 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.041095 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lqwnx" event={"ID":"6405eede-1bde-4bf5-b476-ce244e0029eb","Type":"ContainerStarted","Data":"14236a20bd6c2e9cf2a02465836ed72613e6615b8e0834fdb34268d9e2e2ef48"} Jul 07 20:00:51.619922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.619893 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:51.620088 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.620072 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:51.620324 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.620134 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret podName:6e356e3c-7634-4b62-b81d-46cb0a409d97 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:53.620116155 +0000 UTC m=+6.128788708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret") pod "global-pull-secret-syncer-dsxqc" (UID: "6e356e3c-7634-4b62-b81d-46cb0a409d97") : object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:51.720947 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.720894 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:51.721125 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.721107 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:51.721210 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.721178 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:00:53.721157565 +0000 UTC m=+6.229830139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:51.823030 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.822152 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:51.823030 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.822549 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 20:00:51.823030 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.822569 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 20:00:51.823030 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.822580 2553 projected.go:196] Error preparing data for projected volume kube-api-access-4tdjg for pod openshift-network-diagnostics/network-check-target-47mc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:51.823030 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:51.822642 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg podName:86bcbc7a-039d-4f27-b439-6b6aad64f748 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:53.822618338 +0000 UTC m=+6.331290910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4tdjg" (UniqueName: "kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg") pod "network-check-target-47mc5" (UID: "86bcbc7a-039d-4f27-b439-6b6aad64f748") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:51.969020 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.968967 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 19:55:49 +0000 UTC" deadline="2028-04-24 15:00:27.386844028 +0000 UTC" Jul 07 20:00:51.969020 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:51.969016 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15762h59m35.41783253s" Jul 07 20:00:52.035486 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:52.034135 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:52.035486 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:52.034481 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:00:52.035486 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:52.035168 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:52.035486 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:52.035287 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:00:52.060852 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:52.060820 2553 generic.go:356] "Generic (PLEG): container finished" podID="a1a88a977f1e251bcb667d906b23f249" containerID="50f66a4ef40133336cc853c8f96d8382c74efdf7f1cb251887414f0c606d6eee" exitCode=0 Jul 07 20:00:52.061094 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:52.060868 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" event={"ID":"a1a88a977f1e251bcb667d906b23f249","Type":"ContainerDied","Data":"50f66a4ef40133336cc853c8f96d8382c74efdf7f1cb251887414f0c606d6eee"} Jul 07 20:00:53.023415 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:53.023385 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:53.023861 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.023507 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:00:53.074101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:53.074020 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" event={"ID":"a1a88a977f1e251bcb667d906b23f249","Type":"ContainerStarted","Data":"3efae2fddd9fe463f19453db7ae1aff524ec778deb7c07833d4251e097d49495"} Jul 07 20:00:53.648438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:53.648402 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:53.648643 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.648581 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:53.648719 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.648648 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret podName:6e356e3c-7634-4b62-b81d-46cb0a409d97 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:57.648629563 +0000 UTC m=+10.157302131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret") pod "global-pull-secret-syncer-dsxqc" (UID: "6e356e3c-7634-4b62-b81d-46cb0a409d97") : object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:53.749832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:53.749796 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:53.750049 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.749969 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:53.750049 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.750046 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:00:57.750026496 +0000 UTC m=+10.258699064 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:53.850951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:53.850913 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:53.851140 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.851091 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 20:00:53.851140 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.851113 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 20:00:53.851140 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.851126 2553 projected.go:196] Error preparing data for projected volume kube-api-access-4tdjg for pod openshift-network-diagnostics/network-check-target-47mc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:53.851305 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:53.851185 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg podName:86bcbc7a-039d-4f27-b439-6b6aad64f748 nodeName:}" failed. No retries permitted until 2026-07-07 20:00:57.851167405 +0000 UTC m=+10.359839964 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4tdjg" (UniqueName: "kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg") pod "network-check-target-47mc5" (UID: "86bcbc7a-039d-4f27-b439-6b6aad64f748") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:54.023241 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:54.023165 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:54.023391 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:54.023297 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:00:54.023676 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:54.023657 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:54.024053 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:54.023765 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:00:55.023770 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:55.023286 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:55.023770 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:55.023415 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:00:56.025263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:56.024836 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:56.025263 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:56.024976 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:00:56.029897 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:56.027829 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:56.029897 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:56.028005 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:00:57.023693 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:57.023658 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:57.023883 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.023799 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:00:57.683227 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:57.683184 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:57.683689 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.683358 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:57.683689 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.683436 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret podName:6e356e3c-7634-4b62-b81d-46cb0a409d97 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:05.683416025 +0000 UTC m=+18.192088583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret") pod "global-pull-secret-syncer-dsxqc" (UID: "6e356e3c-7634-4b62-b81d-46cb0a409d97") : object "kube-system"/"original-pull-secret" not registered Jul 07 20:00:57.784004 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:57.783950 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:57.784182 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.784131 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:57.784249 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.784211 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:01:05.78419036 +0000 UTC m=+18.292862922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:00:57.884842 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:57.884367 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:57.884842 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.884544 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 20:00:57.884842 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.884569 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 20:00:57.884842 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.884582 2553 projected.go:196] Error preparing data for projected volume kube-api-access-4tdjg for pod openshift-network-diagnostics/network-check-target-47mc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:57.884842 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:57.884660 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg podName:86bcbc7a-039d-4f27-b439-6b6aad64f748 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:05.884641648 +0000 UTC m=+18.393314204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4tdjg" (UniqueName: "kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg") pod "network-check-target-47mc5" (UID: "86bcbc7a-039d-4f27-b439-6b6aad64f748") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:00:58.023765 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:58.023595 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:00:58.023765 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:58.023709 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:00:58.024167 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:58.024062 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:00:58.024268 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:58.024166 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:00:59.023070 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:00:59.023038 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:00:59.023518 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:00:59.023172 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:00.024008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:00.023956 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:00.024452 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:00.024118 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:00.024658 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:00.024622 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:00.024762 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:00.024736 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:01.023092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:01.023061 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:01.023253 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:01.023181 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:02.023066 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:02.023030 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:02.023066 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:02.023068 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:02.023599 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:02.023174 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:02.023661 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:02.023596 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:03.023208 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:03.023118 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:03.023661 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:03.023270 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:04.023259 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:04.023216 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:04.023628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:04.023216 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:04.023628 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:04.023340 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:04.023628 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:04.023376 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:05.023407 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:05.023377 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:05.023871 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.023478 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:05.754304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:05.754270 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:05.754509 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.754425 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 20:01:05.754571 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.754527 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret podName:6e356e3c-7634-4b62-b81d-46cb0a409d97 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:21.754506776 +0000 UTC m=+34.263179332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret") pod "global-pull-secret-syncer-dsxqc" (UID: "6e356e3c-7634-4b62-b81d-46cb0a409d97") : object "kube-system"/"original-pull-secret" not registered Jul 07 20:01:05.855185 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:05.855062 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:05.855185 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.855176 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:01:05.855396 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.855243 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:01:21.855224438 +0000 UTC m=+34.363896991 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:01:05.955421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:05.955389 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:05.955598 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.955578 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 20:01:05.955665 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.955601 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 20:01:05.955665 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.955613 2553 projected.go:196] Error preparing data for projected volume kube-api-access-4tdjg for pod openshift-network-diagnostics/network-check-target-47mc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:01:05.955759 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:05.955672 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg podName:86bcbc7a-039d-4f27-b439-6b6aad64f748 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:21.955656886 +0000 UTC m=+34.464329452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4tdjg" (UniqueName: "kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg") pod "network-check-target-47mc5" (UID: "86bcbc7a-039d-4f27-b439-6b6aad64f748") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:01:06.023284 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:06.023254 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:06.023454 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:06.023386 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:06.023454 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:06.023424 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:06.023862 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:06.023549 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:07.022969 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:07.022927 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:07.023149 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:07.023064 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:08.024589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:08.024346 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:08.024589 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:08.024473 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:08.024589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:08.024514 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:08.024589 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:08.024581 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:09.023779 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:09.023750 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:09.023945 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:09.023847 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:10.023826 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.023239 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:10.023826 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:10.023368 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:10.023826 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.023438 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:10.023826 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:10.023523 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:10.111099 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.110093 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" event={"ID":"e42e328737f94c0a05dc46b508bb8438","Type":"ContainerStarted","Data":"ed23a16d0c74215b7f60fdf3ec490267574a2b897c568f7ab93bbec0cc5a2e2d"} Jul 07 20:01:10.113054 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.113018 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z27qz" event={"ID":"9817fa91-e60e-4cb5-b49d-40af3b1841a3","Type":"ContainerStarted","Data":"d79647a062a4245bf1fee276d1b2f466783880794f1e601e3ac831ad8003f0ad"} Jul 07 20:01:10.114684 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.114660 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4d29q" event={"ID":"fde5b076-8694-4377-b36d-e5b7656f874f","Type":"ContainerStarted","Data":"e6bcd17869d935e1968ec722c4ce32ab198d23d9067f96a6d399f880ec8f56c0"} Jul 07 20:01:10.116767 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.116612 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerStarted","Data":"206410f7c7fa9e3f93ecb0b3a1c73af59e095a9d28ff513885ad06bb4db1bd3d"} Jul 07 20:01:10.117830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.117789 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t7q6z" event={"ID":"2a31e2bb-5d62-4b23-a428-9943115b4cd3","Type":"ContainerStarted","Data":"5521c710deeb5e5fe468f3abbdc178c91c430fa0670d6ec67a2d833f6617a98a"} Jul 07 20:01:10.119271 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.119141 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" event={"ID":"c42d5506-06d0-4db9-9172-933faebfaee7","Type":"ContainerStarted","Data":"e8a33f67795e828958150a30c464295aa04df42b8fb492e411b5bed9d37872c0"} Jul 07 20:01:10.120857 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.120833 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" event={"ID":"f411591f-909b-46e9-bdd6-812010f8f517","Type":"ContainerStarted","Data":"816dfa618ddee1e738ade1df008179e74b2d71b08b0baa681d6981b88ebdd44b"} Jul 07 20:01:10.123173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.123062 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"51c68453dfed768645c15e56c7fc3db5e374ef59b9f844728e3d3fdbb8b25864"} Jul 07 20:01:10.123173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.123125 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"cb8c54b0b814c4f8f5b72b071dd9db494dff6e7d8e0644ace033f64dfa88400a"} Jul 07 20:01:10.124828 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.124807 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lqwnx" event={"ID":"6405eede-1bde-4bf5-b476-ce244e0029eb","Type":"ContainerStarted","Data":"cec18231932bd76dde5fc88a436b21d899f98efa75688fc2fb9940c3358b6cac"} Jul 07 20:01:10.139017 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.138960 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-86.ec2.internal" podStartSLOduration=21.138949305 podStartE2EDuration="21.138949305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:10.138918651 +0000 UTC m=+22.647591225" watchObservedRunningTime="2026-07-07 20:01:10.138949305 +0000 UTC m=+22.647621881" Jul 07 20:01:10.139226 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.139195 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-86.ec2.internal" podStartSLOduration=21.139190127 podStartE2EDuration="21.139190127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:00:53.097886037 +0000 UTC m=+5.606558613" watchObservedRunningTime="2026-07-07 20:01:10.139190127 +0000 UTC m=+22.647862701" Jul 07 20:01:10.155506 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.155461 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4d29q" podStartSLOduration=6.844860699 podStartE2EDuration="22.155448664s" totalImagesPullingTime="15.310587965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:10.155265705 +0000 UTC m=+22.663938280" watchObservedRunningTime="2026-07-07 20:01:10.155448664 +0000 UTC m=+22.664121238" Jul 07 20:01:10.174949 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.174907 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z27qz" podStartSLOduration=2.542367476 podStartE2EDuration="22.174897601s" totalImagesPullingTime="19.632530125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:10.174530291 +0000 UTC m=+22.683202866" watchObservedRunningTime="2026-07-07 20:01:10.174897601 +0000 UTC m=+22.683570177" Jul 07 20:01:10.192860 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.192820 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lqwnx" podStartSLOduration=6.839936516 podStartE2EDuration="22.192807368s" totalImagesPullingTime="15.352870852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:10.192537615 +0000 UTC m=+22.701210192" watchObservedRunningTime="2026-07-07 20:01:10.192807368 +0000 UTC m=+22.701479943" Jul 07 20:01:10.237740 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.237697 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7fwgt" podStartSLOduration=2.633013167 podStartE2EDuration="22.23768213s" totalImagesPullingTime="19.604668963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:10.236884398 +0000 UTC m=+22.745556973" watchObservedRunningTime="2026-07-07 20:01:10.23768213 +0000 UTC m=+22.746354705" Jul 07 20:01:10.258128 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:10.258090 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t7q6z" podStartSLOduration=2.663666853 podStartE2EDuration="22.258076206s" totalImagesPullingTime="19.594409353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:10.258021769 +0000 UTC m=+22.766694345" watchObservedRunningTime="2026-07-07 20:01:10.258076206 +0000 UTC m=+22.766748780" Jul 07 20:01:11.023733 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.023711 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:11.023851 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:11.023801 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:11.129167 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.129084 2553 generic.go:356] "Generic (PLEG): container finished" podID="bd18ce10-f49a-4a99-96c1-cefbccfb6c80" containerID="206410f7c7fa9e3f93ecb0b3a1c73af59e095a9d28ff513885ad06bb4db1bd3d" exitCode=0 Jul 07 20:01:11.129370 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.129180 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerDied","Data":"206410f7c7fa9e3f93ecb0b3a1c73af59e095a9d28ff513885ad06bb4db1bd3d"} Jul 07 20:01:11.132761 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.132715 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"f1f588644c818757d17b49548e61688f468e09f541884f87c5dadb368453c2b6"} Jul 07 20:01:11.132870 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.132800 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"dbbea9cd5b37c0bd525f9094a3deda764a5b3eda591f24b455a53c1db20c0039"} Jul 07 20:01:11.132870 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.132851 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"1b41aac9f3f612debf7d5d05626e3215641101b7b6f8c566f58a2ec171ff0bd3"} Jul 07 20:01:11.132979 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.132896 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"25e3c2fb1e8ac7eec08d60f46a83b857413b9160f22546b933fe5410983d9ba3"} Jul 07 20:01:11.134182 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.134155 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-grkwb" event={"ID":"2e558b3a-f880-4d8e-84c4-cf3268697b76","Type":"ContainerStarted","Data":"4b3b4274e7f92ea357e6e10c76b577718dd902a4be624d46d944b38ca737a1bb"} Jul 07 20:01:11.183793 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.183751 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-grkwb" podStartSLOduration=7.838087237 podStartE2EDuration="23.183737196s" totalImagesPullingTime="15.345649959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:11.183160523 +0000 UTC m=+23.691833110" watchObservedRunningTime="2026-07-07 20:01:11.183737196 +0000 UTC m=+23.692409772" Jul 07 20:01:11.310716 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.310683 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:01:11.311228 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.311210 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:01:11.951604 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:11.951579 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 20:01:12.022856 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.022714 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T20:01:11.951598804Z","UUID":"a773b19a-914b-4a87-af36-1b9f4f599a16","Handler":null,"Name":"","Endpoint":""} Jul 07 20:01:12.023031 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.022925 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:12.023031 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.022926 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:12.023157 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:12.023064 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:12.023214 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:12.023195 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:12.024338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.024306 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 20:01:12.024338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.024339 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 20:01:12.138413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.138338 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" event={"ID":"f411591f-909b-46e9-bdd6-812010f8f517","Type":"ContainerStarted","Data":"5b05ca5058845528d92395198f0f457c7eea0d08625f03f6b68342b71d07e144"} Jul 07 20:01:12.138608 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.138588 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:01:12.139033 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:12.139017 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4d29q" Jul 07 20:01:13.023718 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:13.023647 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:13.023863 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:13.023744 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:13.147471 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:13.147430 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"f7742bdb4d7559396d8122b9e4b274fb09e5ccffed9eb4a2f3dfb35e8839819c"} Jul 07 20:01:14.023840 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:14.023805 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:14.024010 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:14.023935 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:14.024057 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:14.023986 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:14.024102 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:14.024085 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:14.152401 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:14.152365 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" event={"ID":"f411591f-909b-46e9-bdd6-812010f8f517","Type":"ContainerStarted","Data":"1bdaf80f5755bf11d974ff2c568c6bfdcd8cc1773c951eb45fc5104ddec4b6cb"} Jul 07 20:01:14.180114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:14.180067 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xs542" podStartSLOduration=3.604279914 podStartE2EDuration="26.1800521s" totalImagesPullingTime="22.575772186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:14.178688359 +0000 UTC m=+26.687360955" watchObservedRunningTime="2026-07-07 20:01:14.1800521 +0000 UTC m=+26.688724674" Jul 07 20:01:15.023173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:15.023095 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:15.023310 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:15.023215 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:15.157690 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:15.157657 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" event={"ID":"8297ef12-5367-4138-8cc1-1beba8252179","Type":"ContainerStarted","Data":"c101a7817c465e9ca4eef3fd5839ef53d9970af1542f089836a51e5e28886e87"} Jul 07 20:01:15.204544 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:15.204349 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" podStartSLOduration=7.512497926 podStartE2EDuration="27.204334882s" totalImagesPullingTime="19.691836956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:15.201932646 +0000 UTC m=+27.710605221" watchObservedRunningTime="2026-07-07 20:01:15.204334882 +0000 UTC m=+27.713007456" Jul 07 20:01:16.022962 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.022931 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:16.023149 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:16.023062 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:16.023149 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.023112 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:16.023254 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:16.023198 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:16.160890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.160858 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:01:16.161290 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.161019 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:01:16.161290 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.161086 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:01:16.176607 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.176575 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:01:16.176733 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.176726 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:01:16.824508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.824429 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dt99m"] Jul 07 20:01:16.824672 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.824589 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:16.824741 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:16.824714 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:16.828292 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.828268 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47mc5"] Jul 07 20:01:16.828407 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.828366 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:16.828477 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:16.828457 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:16.850399 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.850373 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dsxqc"] Jul 07 20:01:16.850500 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:16.850450 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:16.850623 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:16.850600 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:18.024038 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:18.024007 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:18.024487 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:18.024105 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:19.023532 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:19.023506 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:19.023627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:19.023516 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:19.023661 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:19.023628 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:19.023733 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:19.023710 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:19.168894 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:19.168865 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerStarted","Data":"4034c5c55011a2af63d1e61ac81c3031a6fb63f718ae82764e055e6faa6e5c00"} Jul 07 20:01:20.023658 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:20.023475 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:20.023819 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:20.023731 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:20.172062 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:20.172029 2553 generic.go:356] "Generic (PLEG): container finished" podID="bd18ce10-f49a-4a99-96c1-cefbccfb6c80" containerID="4034c5c55011a2af63d1e61ac81c3031a6fb63f718ae82764e055e6faa6e5c00" exitCode=0 Jul 07 20:01:20.172389 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:20.172084 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerDied","Data":"4034c5c55011a2af63d1e61ac81c3031a6fb63f718ae82764e055e6faa6e5c00"} Jul 07 20:01:21.022875 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:21.022843 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:21.022875 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:21.022871 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:21.023069 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.022987 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dt99m" podUID="d4d270e6-e127-4b3e-a2c0-a7543f1b09fa" Jul 07 20:01:21.023126 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.023110 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsxqc" podUID="6e356e3c-7634-4b62-b81d-46cb0a409d97" Jul 07 20:01:21.773550 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:21.773512 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:21.774148 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.773642 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 20:01:21.774148 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.773716 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret podName:6e356e3c-7634-4b62-b81d-46cb0a409d97 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:53.773696522 +0000 UTC m=+66.282369081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret") pod "global-pull-secret-syncer-dsxqc" (UID: "6e356e3c-7634-4b62-b81d-46cb0a409d97") : object "kube-system"/"original-pull-secret" not registered Jul 07 20:01:21.874046 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:21.874018 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:21.874213 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.874145 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:01:21.874213 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.874192 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:01:53.874180566 +0000 UTC m=+66.382853122 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 20:01:21.974897 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:21.974856 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:21.975081 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.974970 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 20:01:21.975081 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.974986 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 20:01:21.975081 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.975012 2553 projected.go:196] Error preparing data for projected volume kube-api-access-4tdjg for pod openshift-network-diagnostics/network-check-target-47mc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:01:21.975081 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:21.975056 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg podName:86bcbc7a-039d-4f27-b439-6b6aad64f748 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:53.975043754 +0000 UTC m=+66.483716308 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4tdjg" (UniqueName: "kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg") pod "network-check-target-47mc5" (UID: "86bcbc7a-039d-4f27-b439-6b6aad64f748") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 20:01:22.023437 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.023408 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:22.023641 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:22.023510 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47mc5" podUID="86bcbc7a-039d-4f27-b439-6b6aad64f748" Jul 07 20:01:22.178422 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.178387 2553 generic.go:356] "Generic (PLEG): container finished" podID="bd18ce10-f49a-4a99-96c1-cefbccfb6c80" containerID="8103da377a3b864910179bb9ebd7ce58d4fcab6769386132a421a7389a69e9fa" exitCode=0 Jul 07 20:01:22.178633 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.178436 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerDied","Data":"8103da377a3b864910179bb9ebd7ce58d4fcab6769386132a421a7389a69e9fa"} Jul 07 20:01:22.793927 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.793899 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-86.ec2.internal" event="NodeReady" Jul 07 20:01:22.794479 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.794020 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 20:01:22.838595 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.838565 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-559bc8fb5d-76pcm"] Jul 07 20:01:22.850175 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.850153 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.854212 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.854184 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 20:01:22.854315 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.854275 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 20:01:22.854315 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.854287 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fmqz4\"" Jul 07 20:01:22.855007 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.854976 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 20:01:22.856033 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.856019 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9m5fm"] Jul 07 20:01:22.859319 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.859295 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:22.863504 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.863144 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 20:01:22.865500 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.865271 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 20:01:22.865500 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.865345 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 20:01:22.865500 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.865486 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s6xh9\"" Jul 07 20:01:22.866402 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.866384 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559bc8fb5d-76pcm"] Jul 07 20:01:22.873317 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.873299 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9m5fm"] Jul 07 20:01:22.908512 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.908486 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:22.918391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.918367 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4plnv"] Jul 07 20:01:22.921248 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.921230 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:22.925529 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.925505 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-52pqs\"" Jul 07 20:01:22.925639 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.925621 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 20:01:22.925803 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.925784 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 20:01:22.925900 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.925884 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 20:01:22.934271 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.932179 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4plnv"] Jul 07 20:01:22.982018 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.981984 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-ca-trust-extracted\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982133 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982024 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:22.982133 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982045 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-image-registry-private-configuration\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982133 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982065 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982133 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982114 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-certificates\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982146 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-trusted-ca\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982166 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-bound-sa-token\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982192 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-config-volume\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982212 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-installation-pull-secrets\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982230 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03-cert\") pod \"ingress-canary-4plnv\" (UID: \"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03\") " pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982248 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-tmp-dir\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982277 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7khzf\" (UniqueName: \"kubernetes.io/projected/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-kube-api-access-7khzf\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:22.982326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982312 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjxkf\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-kube-api-access-qjxkf\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:22.982677 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:22.982336 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7llz\" (UniqueName: \"kubernetes.io/projected/b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03-kube-api-access-l7llz\") pod \"ingress-canary-4plnv\" (UID: \"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03\") " pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:23.010793 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.010759 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:23.022837 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.022812 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:23.022936 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.022817 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:23.027147 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.027128 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mcw2j\"" Jul 07 20:01:23.027210 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.027158 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 20:01:23.027210 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.027187 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 20:01:23.083703 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083643 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-installation-pull-secrets\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.083703 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083678 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03-cert\") pod \"ingress-canary-4plnv\" (UID: \"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03\") " pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:23.083703 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083699 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-tmp-dir\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083715 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7khzf\" (UniqueName: \"kubernetes.io/projected/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-kube-api-access-7khzf\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083743 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjxkf\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-kube-api-access-qjxkf\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083758 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7llz\" (UniqueName: \"kubernetes.io/projected/b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03-kube-api-access-l7llz\") pod \"ingress-canary-4plnv\" (UID: \"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03\") " pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083820 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-ca-trust-extracted\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083844 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083869 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-image-registry-private-configuration\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083891 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083914 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-certificates\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083935 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-trusted-ca\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.083951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.083956 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-bound-sa-token\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.084408 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.084019 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-config-volume\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.084408 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.084081 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-tmp-dir\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.084408 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.084346 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 20:01:23.084408 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.084361 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bc8fb5d-76pcm: secret "image-registry-tls" not found Jul 07 20:01:23.084600 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.084418 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls podName:fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:23.584398267 +0000 UTC m=+36.093070843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls") pod "image-registry-559bc8fb5d-76pcm" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92") : secret "image-registry-tls" not found Jul 07 20:01:23.084600 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.084567 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-config-volume\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.084600 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.084579 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-certificates\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.084727 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.084689 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 20:01:23.084767 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.084732 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls podName:7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b nodeName:}" failed. No retries permitted until 2026-07-07 20:01:23.584717852 +0000 UTC m=+36.093390421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls") pod "dns-default-9m5fm" (UID: "7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b") : secret "dns-default-metrics-tls" not found Jul 07 20:01:23.084899 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.084877 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-ca-trust-extracted\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.084946 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.084917 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-trusted-ca\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.087900 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.087876 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-image-registry-private-configuration\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.088035 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.087898 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-installation-pull-secrets\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.088035 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.087885 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03-cert\") pod \"ingress-canary-4plnv\" (UID: \"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03\") " pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:23.095052 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.095022 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-bound-sa-token\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.095376 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.095352 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7khzf\" (UniqueName: \"kubernetes.io/projected/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-kube-api-access-7khzf\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.095485 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.095469 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7llz\" (UniqueName: \"kubernetes.io/projected/b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03-kube-api-access-l7llz\") pod \"ingress-canary-4plnv\" (UID: \"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03\") " pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:23.096709 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.096682 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjxkf\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-kube-api-access-qjxkf\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.234916 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.234886 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4plnv" Jul 07 20:01:23.278247 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.278211 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:23.350974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.350784 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:23.412013 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.411970 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4plnv"] Jul 07 20:01:23.474583 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:23.474558 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ca5bd3_de4d_43a0_b8f9_49d0e24b1b03.slice/crio-796a58b1712f6c4cfd396808d16beff285394975739445b9e9479c6c815c7d12 WatchSource:0}: Error finding container 796a58b1712f6c4cfd396808d16beff285394975739445b9e9479c6c815c7d12: Status 404 returned error can't find the container with id 796a58b1712f6c4cfd396808d16beff285394975739445b9e9479c6c815c7d12 Jul 07 20:01:23.587717 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.587695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:23.587828 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:23.587726 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:23.587828 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.587822 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 20:01:23.588068 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.587837 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bc8fb5d-76pcm: secret "image-registry-tls" not found Jul 07 20:01:23.588068 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.587853 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 20:01:23.588068 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.587883 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls podName:fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:24.587867843 +0000 UTC m=+37.096540402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls") pod "image-registry-559bc8fb5d-76pcm" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92") : secret "image-registry-tls" not found Jul 07 20:01:23.588068 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:23.587909 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls podName:7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b nodeName:}" failed. No retries permitted until 2026-07-07 20:01:24.587891686 +0000 UTC m=+37.096564256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls") pod "dns-default-9m5fm" (UID: "7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b") : secret "dns-default-metrics-tls" not found Jul 07 20:01:24.023865 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.023832 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:24.026790 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.026771 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 20:01:24.027963 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.027948 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 20:01:24.028076 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.027948 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lcjbq\"" Jul 07 20:01:24.185009 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.184971 2553 generic.go:356] "Generic (PLEG): container finished" podID="bd18ce10-f49a-4a99-96c1-cefbccfb6c80" containerID="186cb209fe9bbc5e5b7a436d4883dbc317e5d7bdec5f5413a8d77037279e2782" exitCode=0 Jul 07 20:01:24.185152 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.185054 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerDied","Data":"186cb209fe9bbc5e5b7a436d4883dbc317e5d7bdec5f5413a8d77037279e2782"} Jul 07 20:01:24.186060 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.186038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4plnv" event={"ID":"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03","Type":"ContainerStarted","Data":"796a58b1712f6c4cfd396808d16beff285394975739445b9e9479c6c815c7d12"} Jul 07 20:01:24.595393 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.595360 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:24.595393 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.595394 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:24.595583 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:24.595495 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 20:01:24.595583 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:24.595501 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 20:01:24.595583 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:24.595557 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls podName:7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b nodeName:}" failed. No retries permitted until 2026-07-07 20:01:26.595543506 +0000 UTC m=+39.104216060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls") pod "dns-default-9m5fm" (UID: "7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b") : secret "dns-default-metrics-tls" not found Jul 07 20:01:24.595678 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:24.595507 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bc8fb5d-76pcm: secret "image-registry-tls" not found Jul 07 20:01:24.595678 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:24.595625 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls podName:fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:26.595614076 +0000 UTC m=+39.104286633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls") pod "image-registry-559bc8fb5d-76pcm" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92") : secret "image-registry-tls" not found Jul 07 20:01:24.883445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.883372 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:24.988136 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:24.988105 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:26.613726 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:26.613686 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:26.614132 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:26.613739 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:26.614132 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:26.613854 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 20:01:26.614132 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:26.613857 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 20:01:26.614132 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:26.613915 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bc8fb5d-76pcm: secret "image-registry-tls" not found Jul 07 20:01:26.614132 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:26.613922 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls podName:7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b nodeName:}" failed. No retries permitted until 2026-07-07 20:01:30.613903448 +0000 UTC m=+43.122576020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls") pod "dns-default-9m5fm" (UID: "7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b") : secret "dns-default-metrics-tls" not found Jul 07 20:01:26.614132 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:26.613959 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls podName:fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:30.613946355 +0000 UTC m=+43.122618920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls") pod "image-registry-559bc8fb5d-76pcm" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92") : secret "image-registry-tls" not found Jul 07 20:01:28.144091 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:28.144054 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:28.194971 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:28.194939 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4plnv" event={"ID":"b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03","Type":"ContainerStarted","Data":"a3fb5ab1af92e14ce80aa8a794780368604374773887f7d285e263409de8172c"} Jul 07 20:01:28.214094 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:28.214052 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4plnv" podStartSLOduration=2.571843844 podStartE2EDuration="6.214038036s" totalImagesPullingTime="3.642194192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:28.213349696 +0000 UTC m=+40.722022265" watchObservedRunningTime="2026-07-07 20:01:28.214038036 +0000 UTC m=+40.722710611" Jul 07 20:01:28.224855 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:28.224815 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:30.641212 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:30.641169 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:30.641212 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:30.641216 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:30.641631 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:30.641359 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 20:01:30.641631 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:30.641407 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 20:01:30.641631 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:30.641424 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls podName:7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b nodeName:}" failed. No retries permitted until 2026-07-07 20:01:38.641405706 +0000 UTC m=+51.150078262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls") pod "dns-default-9m5fm" (UID: "7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b") : secret "dns-default-metrics-tls" not found Jul 07 20:01:30.641631 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:30.641428 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bc8fb5d-76pcm: secret "image-registry-tls" not found Jul 07 20:01:30.641631 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:30.641486 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls podName:fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:38.641472818 +0000 UTC m=+51.150145391 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls") pod "image-registry-559bc8fb5d-76pcm" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92") : secret "image-registry-tls" not found Jul 07 20:01:32.881661 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:32.881626 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:32.967767 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:32.967729 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:33.667151 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:33.667116 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:33.769050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:33.769018 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:34.211477 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:34.211447 2553 generic.go:356] "Generic (PLEG): container finished" podID="bd18ce10-f49a-4a99-96c1-cefbccfb6c80" containerID="3121bc199c3010d9ae6a509bc64d59f3d5713b9a0143e55cf832a75a14afba61" exitCode=0 Jul 07 20:01:34.211837 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:34.211510 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerDied","Data":"3121bc199c3010d9ae6a509bc64d59f3d5713b9a0143e55cf832a75a14afba61"} Jul 07 20:01:34.979687 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:34.979648 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:35.145619 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:35.145582 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:35.216960 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:35.216935 2553 generic.go:356] "Generic (PLEG): container finished" podID="bd18ce10-f49a-4a99-96c1-cefbccfb6c80" containerID="e7390bdeaf1fb2a8e342873630440df4113619495470fa2906d8bdcb9b5a35ff" exitCode=0 Jul 07 20:01:35.217305 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:35.217008 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerDied","Data":"e7390bdeaf1fb2a8e342873630440df4113619495470fa2906d8bdcb9b5a35ff"} Jul 07 20:01:35.942439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:35.942404 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:36.043067 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:36.043017 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:36.222184 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:36.222111 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjj97" event={"ID":"bd18ce10-f49a-4a99-96c1-cefbccfb6c80","Type":"ContainerStarted","Data":"7f01fe355429243c2acf39477156ab91367918b5505d8df3351beb01075a7918"} Jul 07 20:01:36.415619 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:36.415573 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:36.500079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:36.500047 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:38.703182 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:38.703142 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:38.703182 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:38.703185 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:38.703659 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:38.703289 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 20:01:38.703659 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:38.703289 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 20:01:38.703659 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:38.703354 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls podName:7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b nodeName:}" failed. No retries permitted until 2026-07-07 20:01:54.70333791 +0000 UTC m=+67.212010463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls") pod "dns-default-9m5fm" (UID: "7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b") : secret "dns-default-metrics-tls" not found Jul 07 20:01:38.703659 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:38.703299 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bc8fb5d-76pcm: secret "image-registry-tls" not found Jul 07 20:01:38.703659 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:38.703390 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls podName:fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:54.703384055 +0000 UTC m=+67.212056656 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls") pod "image-registry-559bc8fb5d-76pcm" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92") : secret "image-registry-tls" not found Jul 07 20:01:39.871414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:39.871376 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:41.551405 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.551347 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vjj97" podStartSLOduration=14.62687034 podStartE2EDuration="53.551329483s" totalImagesPullingTime="38.924459143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:36.254680993 +0000 UTC m=+48.763353565" watchObservedRunningTime="2026-07-07 20:01:41.551329483 +0000 UTC m=+54.060002057" Jul 07 20:01:41.551872 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.551750 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t"] Jul 07 20:01:41.553848 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.553829 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk"] Jul 07 20:01:41.553973 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.553943 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.556125 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.556108 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p"] Jul 07 20:01:41.556232 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.556214 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" Jul 07 20:01:41.558663 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.558645 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:41.560824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.560806 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 20:01:41.561016 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.560982 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 20:01:41.561016 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.561007 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.561266 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.561155 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.561343 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.561292 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hk2d7\"" Jul 07 20:01:41.561343 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.561333 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.562278 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.562247 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-njb64\"" Jul 07 20:01:41.562278 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.562250 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.565656 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.565641 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hwq5h\"" Jul 07 20:01:41.566020 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.566003 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.569058 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.569020 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 20:01:41.569265 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.569244 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.570434 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.570409 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t"] Jul 07 20:01:41.571305 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.571282 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk"] Jul 07 20:01:41.571956 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.571897 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p"] Jul 07 20:01:41.697615 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.697582 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-shj6l"] Jul 07 20:01:41.700103 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.700088 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.717008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.716976 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.717138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.717047 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 20:01:41.717138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.717094 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 20:01:41.718518 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.718504 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.724983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.724963 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gf67w\"" Jul 07 20:01:41.727190 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.727173 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2rzk\" (UniqueName: \"kubernetes.io/projected/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-kube-api-access-r2rzk\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:41.727254 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.727201 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.727289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.727259 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhvv2\" (UniqueName: \"kubernetes.io/projected/fb2867af-3417-4f9d-9fc6-d4a95a373462-kube-api-access-nhvv2\") pod \"volume-data-source-validator-5d456fdbcd-kdtlk\" (UID: \"fb2867af-3417-4f9d-9fc6-d4a95a373462\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" Jul 07 20:01:41.727289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.727277 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.727368 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.727293 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:41.727368 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.727318 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvlsb\" (UniqueName: \"kubernetes.io/projected/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-kube-api-access-gvlsb\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.728736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.728719 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-shj6l"] Jul 07 20:01:41.740776 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.740757 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 20:01:41.801750 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.801686 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cqz4g"] Jul 07 20:01:41.804273 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.804259 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:41.808393 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.808376 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb"] Jul 07 20:01:41.810855 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.810839 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:41.811405 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.811388 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 20:01:41.826339 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.826318 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.826689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.826676 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fzzr2\"" Jul 07 20:01:41.827223 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.827201 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 20:01:41.827818 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.827780 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.829471 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829444 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhvv2\" (UniqueName: \"kubernetes.io/projected/fb2867af-3417-4f9d-9fc6-d4a95a373462-kube-api-access-nhvv2\") pod \"volume-data-source-validator-5d456fdbcd-kdtlk\" (UID: \"fb2867af-3417-4f9d-9fc6-d4a95a373462\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" Jul 07 20:01:41.829574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829495 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.829574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829531 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:41.829574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829564 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7930caa-d305-4b26-985f-a89a0abb6b08-serving-cert\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.829790 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829603 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvlsb\" (UniqueName: \"kubernetes.io/projected/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-kube-api-access-gvlsb\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.829790 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829714 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2rzk\" (UniqueName: \"kubernetes.io/projected/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-kube-api-access-r2rzk\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:41.829790 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829742 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7930caa-d305-4b26-985f-a89a0abb6b08-config\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.829790 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829777 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.829966 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829808 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7930caa-d305-4b26-985f-a89a0abb6b08-trusted-ca\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.829966 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.829839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtsjz\" (UniqueName: \"kubernetes.io/projected/b7930caa-d305-4b26-985f-a89a0abb6b08-kube-api-access-mtsjz\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.830714 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.830689 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.830840 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:41.830820 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 20:01:41.830918 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:41.830906 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls podName:b37c9acc-aeb3-4b6e-8e78-209c1d8227bc nodeName:}" failed. No retries permitted until 2026-07-07 20:01:42.330888397 +0000 UTC m=+54.839560965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-wcx8p" (UID: "b37c9acc-aeb3-4b6e-8e78-209c1d8227bc") : secret "samples-operator-tls" not found Jul 07 20:01:41.830981 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.830962 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x"] Jul 07 20:01:41.834068 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.834050 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-skstg"] Jul 07 20:01:41.834185 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.834169 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:41.834740 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.834722 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.836407 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.836376 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.836483 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.836379 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 20:01:41.836483 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.836385 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 20:01:41.836593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.836551 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" Jul 07 20:01:41.837315 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.837298 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-757c6f7d89-5mjdr"] Jul 07 20:01:41.839707 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.839689 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:41.841017 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.840975 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vl7sf\"" Jul 07 20:01:41.841108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.841031 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.853792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.853775 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb"] Jul 07 20:01:41.871862 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.871844 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 20:01:41.871970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.871863 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h82kq\"" Jul 07 20:01:41.872819 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.872802 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cqz4g"] Jul 07 20:01:41.874781 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.874760 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhvv2\" (UniqueName: \"kubernetes.io/projected/fb2867af-3417-4f9d-9fc6-d4a95a373462-kube-api-access-nhvv2\") pod \"volume-data-source-validator-5d456fdbcd-kdtlk\" (UID: \"fb2867af-3417-4f9d-9fc6-d4a95a373462\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" Jul 07 20:01:41.878498 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.878481 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 20:01:41.879312 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.879296 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-skstg"] Jul 07 20:01:41.881126 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.881100 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-757c6f7d89-5mjdr"] Jul 07 20:01:41.882480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.882462 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.885417 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.885401 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 20:01:41.885899 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.885876 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 20:01:41.885984 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.885921 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 20:01:41.885984 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.885943 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 20:01:41.886116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.886031 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-k62z7\"" Jul 07 20:01:41.886116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.886064 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.886221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.886158 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j6m9q\"" Jul 07 20:01:41.886308 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.886281 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 20:01:41.887201 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.887184 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 20:01:41.888085 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.888066 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2rzk\" (UniqueName: \"kubernetes.io/projected/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-kube-api-access-r2rzk\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:41.892735 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.892707 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvlsb\" (UniqueName: \"kubernetes.io/projected/d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a-kube-api-access-gvlsb\") pod \"kube-storage-version-migrator-operator-846bf668f7-8795t\" (UID: \"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:41.901558 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.901539 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 20:01:41.903683 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.903663 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x"] Jul 07 20:01:41.930738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.930711 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvtng\" (UniqueName: \"kubernetes.io/projected/7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0-kube-api-access-kvtng\") pod \"network-check-source-989f8b8dc-skstg\" (UID: \"7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" Jul 07 20:01:41.930843 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.930744 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-tmp\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:41.930843 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.930775 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7930caa-d305-4b26-985f-a89a0abb6b08-serving-cert\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.930961 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.930849 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:41.930961 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.930929 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:41.931095 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931008 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cac9670-2657-43c9-a27a-a12fd7f54db2-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:41.931095 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931040 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:41.931095 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931088 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-stats-auth\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:41.931240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931120 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-serving-cert\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:41.931240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931179 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx2pk\" (UniqueName: \"kubernetes.io/projected/beb9ec18-4e3a-4655-a6db-6313d35fca26-kube-api-access-cx2pk\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:41.931240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931205 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:41.931240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931223 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jbk8\" (UniqueName: \"kubernetes.io/projected/a73237eb-e751-46a4-ad48-17242e2dd132-kube-api-access-6jbk8\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:41.931430 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931255 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:41.931430 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931296 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw5qw\" (UniqueName: \"kubernetes.io/projected/1cac9670-2657-43c9-a27a-a12fd7f54db2-kube-api-access-vw5qw\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:41.931430 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931353 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7930caa-d305-4b26-985f-a89a0abb6b08-config\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.931430 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931372 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p8jd\" (UniqueName: \"kubernetes.io/projected/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-kube-api-access-8p8jd\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:41.931627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931487 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtsjz\" (UniqueName: \"kubernetes.io/projected/b7930caa-d305-4b26-985f-a89a0abb6b08-kube-api-access-mtsjz\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.931627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931512 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7930caa-d305-4b26-985f-a89a0abb6b08-trusted-ca\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.931627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931544 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cac9670-2657-43c9-a27a-a12fd7f54db2-config\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:41.931627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931578 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/beb9ec18-4e3a-4655-a6db-6313d35fca26-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:41.931627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931594 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-default-certificate\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:41.931627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.931624 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-snapshots\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:41.932097 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.932076 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7930caa-d305-4b26-985f-a89a0abb6b08-config\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.932378 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.932363 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b7930caa-d305-4b26-985f-a89a0abb6b08-trusted-ca\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.933118 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.933097 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7930caa-d305-4b26-985f-a89a0abb6b08-serving-cert\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:41.940975 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:41.940949 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtsjz\" (UniqueName: \"kubernetes.io/projected/b7930caa-d305-4b26-985f-a89a0abb6b08-kube-api-access-mtsjz\") pod \"console-operator-68b757fb67-shj6l\" (UID: \"b7930caa-d305-4b26-985f-a89a0abb6b08\") " pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:42.008480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.008453 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:42.032406 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032376 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx2pk\" (UniqueName: \"kubernetes.io/projected/beb9ec18-4e3a-4655-a6db-6313d35fca26-kube-api-access-cx2pk\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:42.032536 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032430 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.032536 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032459 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jbk8\" (UniqueName: \"kubernetes.io/projected/a73237eb-e751-46a4-ad48-17242e2dd132-kube-api-access-6jbk8\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.032536 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032487 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:42.032536 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032504 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw5qw\" (UniqueName: \"kubernetes.io/projected/1cac9670-2657-43c9-a27a-a12fd7f54db2-kube-api-access-vw5qw\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:42.032735 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.032718 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:42.032783 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032733 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p8jd\" (UniqueName: \"kubernetes.io/projected/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-kube-api-access-8p8jd\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.032838 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.032804 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls podName:beb9ec18-4e3a-4655-a6db-6313d35fca26 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:42.532783248 +0000 UTC m=+55.041455805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x9f9x" (UID: "beb9ec18-4e3a-4655-a6db-6313d35fca26") : secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:42.032900 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032852 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cac9670-2657-43c9-a27a-a12fd7f54db2-config\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:42.032953 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032895 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/beb9ec18-4e3a-4655-a6db-6313d35fca26-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:42.032953 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032921 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-default-certificate\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.033073 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032954 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-snapshots\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.033073 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.032983 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvtng\" (UniqueName: \"kubernetes.io/projected/7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0-kube-api-access-kvtng\") pod \"network-check-source-989f8b8dc-skstg\" (UID: \"7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" Jul 07 20:01:42.033073 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033034 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-tmp\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.033073 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033065 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.033267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033090 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.033267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033117 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.033267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033146 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cac9670-2657-43c9-a27a-a12fd7f54db2-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:42.033267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033174 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.033267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033204 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-stats-auth\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.033267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033234 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-serving-cert\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.033555 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033375 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-tmp\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.033555 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033519 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cac9670-2657-43c9-a27a-a12fd7f54db2-config\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:42.033651 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.033625 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 20:01:42.033701 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033663 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-snapshots\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.033701 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.033683 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:42.533666716 +0000 UTC m=+55.042339285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : secret "router-metrics-certs-default" not found Jul 07 20:01:42.033792 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.033702 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:42.533692891 +0000 UTC m=+55.042365450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : configmap references non-existent config key: service-ca.crt Jul 07 20:01:42.033792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.033680 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/beb9ec18-4e3a-4655-a6db-6313d35fca26-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:42.034171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.034151 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.035972 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.035953 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cac9670-2657-43c9-a27a-a12fd7f54db2-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:42.036184 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.036166 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-serving-cert\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.036288 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.036272 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-default-certificate\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.036375 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.036356 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-stats-auth\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.047683 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.047636 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx2pk\" (UniqueName: \"kubernetes.io/projected/beb9ec18-4e3a-4655-a6db-6313d35fca26-kube-api-access-cx2pk\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:42.048713 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.048694 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvtng\" (UniqueName: \"kubernetes.io/projected/7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0-kube-api-access-kvtng\") pod \"network-check-source-989f8b8dc-skstg\" (UID: \"7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" Jul 07 20:01:42.051592 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.051568 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jbk8\" (UniqueName: \"kubernetes.io/projected/a73237eb-e751-46a4-ad48-17242e2dd132-kube-api-access-6jbk8\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.052519 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.052472 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p8jd\" (UniqueName: \"kubernetes.io/projected/d2bcb0b9-8e9b-4cd1-9633-853e6668363e-kube-api-access-8p8jd\") pod \"insights-operator-5c847dcfc4-cqz4g\" (UID: \"d2bcb0b9-8e9b-4cd1-9633-853e6668363e\") " pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.056445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.056411 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw5qw\" (UniqueName: \"kubernetes.io/projected/1cac9670-2657-43c9-a27a-a12fd7f54db2-kube-api-access-vw5qw\") pod \"service-ca-operator-5d46d9ffd4-7qwcb\" (UID: \"1cac9670-2657-43c9-a27a-a12fd7f54db2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:42.112343 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.112308 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" Jul 07 20:01:42.119461 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.119443 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" Jul 07 20:01:42.134904 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.134871 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-shj6l"] Jul 07 20:01:42.139928 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:42.139902 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7930caa_d305_4b26_985f_a89a0abb6b08.slice/crio-50c149f12c8aed648ebb2353c0582905d6d2b0d651436bf7768e5f59028ddb48 WatchSource:0}: Error finding container 50c149f12c8aed648ebb2353c0582905d6d2b0d651436bf7768e5f59028ddb48: Status 404 returned error can't find the container with id 50c149f12c8aed648ebb2353c0582905d6d2b0d651436bf7768e5f59028ddb48 Jul 07 20:01:42.154612 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.154591 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" Jul 07 20:01:42.163719 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.163695 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" Jul 07 20:01:42.173620 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.173424 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" Jul 07 20:01:42.241326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.241166 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" event={"ID":"b7930caa-d305-4b26-985f-a89a0abb6b08","Type":"ContainerStarted","Data":"50c149f12c8aed648ebb2353c0582905d6d2b0d651436bf7768e5f59028ddb48"} Jul 07 20:01:42.291842 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.291808 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cqz4g"] Jul 07 20:01:42.296403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.295601 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb"] Jul 07 20:01:42.299737 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:42.299714 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cac9670_2657_43c9_a27a_a12fd7f54db2.slice/crio-8a2e94417d3a26ceeaa6d14d5ba5d1d0e9fe56d13c3b09e8b65e5f866d027159 WatchSource:0}: Error finding container 8a2e94417d3a26ceeaa6d14d5ba5d1d0e9fe56d13c3b09e8b65e5f866d027159: Status 404 returned error can't find the container with id 8a2e94417d3a26ceeaa6d14d5ba5d1d0e9fe56d13c3b09e8b65e5f866d027159 Jul 07 20:01:42.331568 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.331542 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-skstg"] Jul 07 20:01:42.334404 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:42.334378 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c05ff8a_3739_4d1a_9b32_3fe0d29d98c0.slice/crio-85af4ad06b0098f2aae3bed1b2a29b6d50c041cf0e8ea37bade907e23d71d808 WatchSource:0}: Error finding container 85af4ad06b0098f2aae3bed1b2a29b6d50c041cf0e8ea37bade907e23d71d808: Status 404 returned error can't find the container with id 85af4ad06b0098f2aae3bed1b2a29b6d50c041cf0e8ea37bade907e23d71d808 Jul 07 20:01:42.336361 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.336334 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:42.336489 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.336473 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 20:01:42.336628 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.336615 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls podName:b37c9acc-aeb3-4b6e-8e78-209c1d8227bc nodeName:}" failed. No retries permitted until 2026-07-07 20:01:43.33658925 +0000 UTC m=+55.845261804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-wcx8p" (UID: "b37c9acc-aeb3-4b6e-8e78-209c1d8227bc") : secret "samples-operator-tls" not found Jul 07 20:01:42.355691 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.355670 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t"] Jul 07 20:01:42.358197 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:42.358177 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3fcf2c5_7c84_4931_b2f7_b64c956b4a0a.slice/crio-3b9a6430564b1152b68e845535f089c38d31fa623efd2ab55714d6e620156f5b WatchSource:0}: Error finding container 3b9a6430564b1152b68e845535f089c38d31fa623efd2ab55714d6e620156f5b: Status 404 returned error can't find the container with id 3b9a6430564b1152b68e845535f089c38d31fa623efd2ab55714d6e620156f5b Jul 07 20:01:42.363785 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.363765 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk"] Jul 07 20:01:42.366668 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:42.366648 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2867af_3417_4f9d_9fc6_d4a95a373462.slice/crio-3fe9d1f8741412efad6880b09990fe456382043916f4323420fce6d5e9c7e7b9 WatchSource:0}: Error finding container 3fe9d1f8741412efad6880b09990fe456382043916f4323420fce6d5e9c7e7b9: Status 404 returned error can't find the container with id 3fe9d1f8741412efad6880b09990fe456382043916f4323420fce6d5e9c7e7b9 Jul 07 20:01:42.538479 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.538442 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:42.538628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.538544 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.538628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.538568 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:42.538628 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.538591 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:42.538719 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.538655 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls podName:beb9ec18-4e3a-4655-a6db-6313d35fca26 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:43.538639475 +0000 UTC m=+56.047312030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x9f9x" (UID: "beb9ec18-4e3a-4655-a6db-6313d35fca26") : secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:42.538719 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.538671 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:43.538664839 +0000 UTC m=+56.047337392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : configmap references non-existent config key: service-ca.crt Jul 07 20:01:42.538719 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.538673 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 20:01:42.538719 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:42.538715 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:43.538702335 +0000 UTC m=+56.047374887 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : secret "router-metrics-certs-default" not found Jul 07 20:01:42.904091 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.904053 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:42.984053 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:42.984006 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:43.247237 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.247141 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" event={"ID":"fb2867af-3417-4f9d-9fc6-d4a95a373462","Type":"ContainerStarted","Data":"3fe9d1f8741412efad6880b09990fe456382043916f4323420fce6d5e9c7e7b9"} Jul 07 20:01:43.249266 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.249190 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" event={"ID":"1cac9670-2657-43c9-a27a-a12fd7f54db2","Type":"ContainerStarted","Data":"8a2e94417d3a26ceeaa6d14d5ba5d1d0e9fe56d13c3b09e8b65e5f866d027159"} Jul 07 20:01:43.251046 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.251019 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" event={"ID":"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a","Type":"ContainerStarted","Data":"3b9a6430564b1152b68e845535f089c38d31fa623efd2ab55714d6e620156f5b"} Jul 07 20:01:43.253049 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.253015 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" event={"ID":"d2bcb0b9-8e9b-4cd1-9633-853e6668363e","Type":"ContainerStarted","Data":"13d2f4aba0df5f5cac7b0b3ae7fd540b960616386ad2b0f3c2f2e517041b8f6a"} Jul 07 20:01:43.254308 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.254285 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" event={"ID":"7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0","Type":"ContainerStarted","Data":"85af4ad06b0098f2aae3bed1b2a29b6d50c041cf0e8ea37bade907e23d71d808"} Jul 07 20:01:43.344332 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.344296 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:43.344482 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:43.344452 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 20:01:43.344541 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:43.344524 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls podName:b37c9acc-aeb3-4b6e-8e78-209c1d8227bc nodeName:}" failed. No retries permitted until 2026-07-07 20:01:45.344505224 +0000 UTC m=+57.853177780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-wcx8p" (UID: "b37c9acc-aeb3-4b6e-8e78-209c1d8227bc") : secret "samples-operator-tls" not found Jul 07 20:01:43.546977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.546895 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:43.546977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.546952 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:43.547222 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:43.547063 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:43.547222 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:43.547103 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:45.547081422 +0000 UTC m=+58.055753976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : configmap references non-existent config key: service-ca.crt Jul 07 20:01:43.547222 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:43.547179 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls podName:beb9ec18-4e3a-4655-a6db-6313d35fca26 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:45.547156283 +0000 UTC m=+58.055828850 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x9f9x" (UID: "beb9ec18-4e3a-4655-a6db-6313d35fca26") : secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:43.547375 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:43.547229 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:43.547375 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:43.547323 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 20:01:43.547375 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:43.547362 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:45.547350919 +0000 UTC m=+58.056023476 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : secret "router-metrics-certs-default" not found Jul 07 20:01:44.877596 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:44.877557 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-54pwj"] Jul 07 20:01:44.881089 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:44.881063 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:44.885313 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:44.885287 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mhhnd\"" Jul 07 20:01:44.885588 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:44.885565 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 20:01:44.886938 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:44.886919 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 20:01:44.916582 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:44.916499 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-54pwj"] Jul 07 20:01:45.062893 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.062860 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a400cf5f-3487-46ce-8351-f46672ac0747-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:45.063102 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.062933 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:45.163970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.163887 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a400cf5f-3487-46ce-8351-f46672ac0747-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:45.164157 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.164108 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:45.164968 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.164474 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 20:01:45.164968 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.164545 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert podName:a400cf5f-3487-46ce-8351-f46672ac0747 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:45.664524911 +0000 UTC m=+58.173197470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-54pwj" (UID: "a400cf5f-3487-46ce-8351-f46672ac0747") : secret "networking-console-plugin-cert" not found Jul 07 20:01:45.165307 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.165282 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a400cf5f-3487-46ce-8351-f46672ac0747-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:45.280044 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.279970 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:45.366073 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.366039 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:45.366239 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.366156 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 20:01:45.366239 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.366217 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls podName:b37c9acc-aeb3-4b6e-8e78-209c1d8227bc nodeName:}" failed. No retries permitted until 2026-07-07 20:01:49.366202428 +0000 UTC m=+61.874874983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-wcx8p" (UID: "b37c9acc-aeb3-4b6e-8e78-209c1d8227bc") : secret "samples-operator-tls" not found Jul 07 20:01:45.390199 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.390158 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:45.568348 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.568311 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:45.568517 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.568371 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:45.568517 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.568417 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:45.568517 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.568485 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 20:01:45.568703 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.568564 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:49.568542417 +0000 UTC m=+62.077214987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : secret "router-metrics-certs-default" not found Jul 07 20:01:45.568703 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.568585 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:49.568574416 +0000 UTC m=+62.077246975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : configmap references non-existent config key: service-ca.crt Jul 07 20:01:45.568703 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.568594 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:45.568703 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.568663 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls podName:beb9ec18-4e3a-4655-a6db-6313d35fca26 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:49.568647134 +0000 UTC m=+62.077319692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x9f9x" (UID: "beb9ec18-4e3a-4655-a6db-6313d35fca26") : secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:45.669111 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:45.669075 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:45.669366 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.669346 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 20:01:45.669492 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:45.669416 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert podName:a400cf5f-3487-46ce-8351-f46672ac0747 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:46.669395276 +0000 UTC m=+59.178067847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-54pwj" (UID: "a400cf5f-3487-46ce-8351-f46672ac0747") : secret "networking-console-plugin-cert" not found Jul 07 20:01:46.677608 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:46.677564 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:46.678045 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:46.677750 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 20:01:46.678045 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:46.677836 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert podName:a400cf5f-3487-46ce-8351-f46672ac0747 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:48.677813921 +0000 UTC m=+61.186486493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-54pwj" (UID: "a400cf5f-3487-46ce-8351-f46672ac0747") : secret "networking-console-plugin-cert" not found Jul 07 20:01:47.989116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:47.989071 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:48.138222 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.138100 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:48.178129 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.178096 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tk7bx" Jul 07 20:01:48.270936 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.270463 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" event={"ID":"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a","Type":"ContainerStarted","Data":"a7d7ccc97ac65a3f2db8c69b7956535b977389c8dceb700d5022bd7ebed1a7ea"} Jul 07 20:01:48.272234 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.272206 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" event={"ID":"d2bcb0b9-8e9b-4cd1-9633-853e6668363e","Type":"ContainerStarted","Data":"1418b007cef14aa686d57fd6a49cdcec4d44078a34cf0452b4672040d0e2c398"} Jul 07 20:01:48.274063 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.274013 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" event={"ID":"7c05ff8a-3739-4d1a-9b32-3fe0d29d98c0","Type":"ContainerStarted","Data":"980bc0b34ce8b96294b1dd524ad62ada774369af246a3cd5dc99db634dca33b1"} Jul 07 20:01:48.275755 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.275734 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/0.log" Jul 07 20:01:48.275844 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.275768 2553 generic.go:356] "Generic (PLEG): container finished" podID="b7930caa-d305-4b26-985f-a89a0abb6b08" containerID="757ddd25898f49dee15eb92743ccf8d282ad27f4fe37e99ff5e09745ec122e14" exitCode=255 Jul 07 20:01:48.275900 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.275835 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" event={"ID":"b7930caa-d305-4b26-985f-a89a0abb6b08","Type":"ContainerDied","Data":"757ddd25898f49dee15eb92743ccf8d282ad27f4fe37e99ff5e09745ec122e14"} Jul 07 20:01:48.276831 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.276140 2553 scope.go:117] "RemoveContainer" containerID="757ddd25898f49dee15eb92743ccf8d282ad27f4fe37e99ff5e09745ec122e14" Jul 07 20:01:48.279155 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.278764 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" event={"ID":"fb2867af-3417-4f9d-9fc6-d4a95a373462","Type":"ContainerStarted","Data":"54367bb739736b35b2fc465f142ae79427fee88b2d70906478a32423febc901d"} Jul 07 20:01:48.281269 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.280809 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" event={"ID":"1cac9670-2657-43c9-a27a-a12fd7f54db2","Type":"ContainerStarted","Data":"0c48d69728bb04ebeb0c891001c8570a6a21aaa1527ff55f8fa053e1f633e87f"} Jul 07 20:01:48.300083 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.300045 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" podStartSLOduration=1.655170839 podStartE2EDuration="7.30003195s" totalImagesPullingTime="5.644861111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:48.299673449 +0000 UTC m=+60.808346022" watchObservedRunningTime="2026-07-07 20:01:48.30003195 +0000 UTC m=+60.808704524" Jul 07 20:01:48.360384 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.359912 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-skstg" podStartSLOduration=1.6468393479999999 podStartE2EDuration="7.359891529s" totalImagesPullingTime="5.713052181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:48.358491267 +0000 UTC m=+60.867163858" watchObservedRunningTime="2026-07-07 20:01:48.359891529 +0000 UTC m=+60.868564106" Jul 07 20:01:48.384554 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.383702 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kdtlk" podStartSLOduration=1.754359845 podStartE2EDuration="7.383685558s" totalImagesPullingTime="5.629325713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:48.383011646 +0000 UTC m=+60.891684222" watchObservedRunningTime="2026-07-07 20:01:48.383685558 +0000 UTC m=+60.892358136" Jul 07 20:01:48.396641 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.396616 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:48.404808 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.404768 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" podStartSLOduration=1.703991952 podStartE2EDuration="7.404752645s" totalImagesPullingTime="5.700760693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:48.403540248 +0000 UTC m=+60.912212824" watchObservedRunningTime="2026-07-07 20:01:48.404752645 +0000 UTC m=+60.913425220" Jul 07 20:01:48.426947 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.426897 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" podStartSLOduration=1.727545084 podStartE2EDuration="7.426877076s" totalImagesPullingTime="5.699331992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:48.425343914 +0000 UTC m=+60.934016492" watchObservedRunningTime="2026-07-07 20:01:48.426877076 +0000 UTC m=+60.935549653" Jul 07 20:01:48.524438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.524358 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:48.696123 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:48.696083 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:48.696296 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:48.696250 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 20:01:48.696372 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:48.696327 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert podName:a400cf5f-3487-46ce-8351-f46672ac0747 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:52.696306252 +0000 UTC m=+65.204978808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-54pwj" (UID: "a400cf5f-3487-46ce-8351-f46672ac0747") : secret "networking-console-plugin-cert" not found Jul 07 20:01:49.293222 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.293151 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/1.log" Jul 07 20:01:49.293686 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.293627 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/0.log" Jul 07 20:01:49.293686 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.293668 2553 generic.go:356] "Generic (PLEG): container finished" podID="b7930caa-d305-4b26-985f-a89a0abb6b08" containerID="da09e563e1a2175659f01fea7e15f5f71b5d7120b99619e091a0dd01d4fd57a5" exitCode=255 Jul 07 20:01:49.293835 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.293746 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" event={"ID":"b7930caa-d305-4b26-985f-a89a0abb6b08","Type":"ContainerDied","Data":"da09e563e1a2175659f01fea7e15f5f71b5d7120b99619e091a0dd01d4fd57a5"} Jul 07 20:01:49.293922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.293907 2553 scope.go:117] "RemoveContainer" containerID="757ddd25898f49dee15eb92743ccf8d282ad27f4fe37e99ff5e09745ec122e14" Jul 07 20:01:49.294327 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.294293 2553 scope.go:117] "RemoveContainer" containerID="da09e563e1a2175659f01fea7e15f5f71b5d7120b99619e091a0dd01d4fd57a5" Jul 07 20:01:49.294644 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.294618 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-shj6l_openshift-console-operator(b7930caa-d305-4b26-985f-a89a0abb6b08)\"" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" podUID="b7930caa-d305-4b26-985f-a89a0abb6b08" Jul 07 20:01:49.352119 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.352076 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:49.403072 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.403037 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:49.403439 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.403407 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 20:01:49.403554 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.403472 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls podName:b37c9acc-aeb3-4b6e-8e78-209c1d8227bc nodeName:}" failed. No retries permitted until 2026-07-07 20:01:57.403453668 +0000 UTC m=+69.912126231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-wcx8p" (UID: "b37c9acc-aeb3-4b6e-8e78-209c1d8227bc") : secret "samples-operator-tls" not found Jul 07 20:01:49.460101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.460067 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:49.560342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.560260 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:49.605728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.605695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:49.605887 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.605749 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:49.605887 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:49.605798 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:49.605887 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.605852 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 20:01:49.606060 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.605928 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:57.605908108 +0000 UTC m=+70.114580669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : secret "router-metrics-certs-default" not found Jul 07 20:01:49.606060 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.605934 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:49.606060 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.605948 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:57.605938526 +0000 UTC m=+70.114611080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : configmap references non-existent config key: service-ca.crt Jul 07 20:01:49.606060 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:49.606013 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls podName:beb9ec18-4e3a-4655-a6db-6313d35fca26 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:57.605973226 +0000 UTC m=+70.114645785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x9f9x" (UID: "beb9ec18-4e3a-4655-a6db-6313d35fca26") : secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:50.040045 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.040013 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s6npk"] Jul 07 20:01:50.044951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.044924 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.048530 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.048495 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 20:01:50.048935 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.048912 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b4fhd\"" Jul 07 20:01:50.049227 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.049207 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 20:01:50.059432 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.059395 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s6npk"] Jul 07 20:01:50.099262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.099231 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:50.111304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.111278 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08eeb706-817f-4562-859c-f42a542511a3-crio-socket\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.111416 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.111309 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.111456 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.111420 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08eeb706-817f-4562-859c-f42a542511a3-data-volume\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.111490 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.111456 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.111490 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.111481 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwzcd\" (UniqueName: \"kubernetes.io/projected/08eeb706-817f-4562-859c-f42a542511a3-kube-api-access-kwzcd\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.208838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.208802 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:50.212826 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.212792 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwzcd\" (UniqueName: \"kubernetes.io/projected/08eeb706-817f-4562-859c-f42a542511a3-kube-api-access-kwzcd\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.212932 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.212919 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08eeb706-817f-4562-859c-f42a542511a3-crio-socket\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.212986 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.212943 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.213057 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.213045 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08eeb706-817f-4562-859c-f42a542511a3-data-volume\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.213108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.213059 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08eeb706-817f-4562-859c-f42a542511a3-crio-socket\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.213108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.213076 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.213208 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.213159 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 20:01:50.213208 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.213197 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 20:01:50.213294 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.213212 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:50.713197526 +0000 UTC m=+63.221870098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : secret "insights-runtime-extractor-tls" not found Jul 07 20:01:50.213294 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.213255 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:50.713238164 +0000 UTC m=+63.221910737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : configmap "kube-rbac-proxy" not found Jul 07 20:01:50.213448 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.213431 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08eeb706-817f-4562-859c-f42a542511a3-data-volume\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.228306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.228279 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwzcd\" (UniqueName: \"kubernetes.io/projected/08eeb706-817f-4562-859c-f42a542511a3-kube-api-access-kwzcd\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.299147 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.299079 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/1.log" Jul 07 20:01:50.299577 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.299474 2553 scope.go:117] "RemoveContainer" containerID="da09e563e1a2175659f01fea7e15f5f71b5d7120b99619e091a0dd01d4fd57a5" Jul 07 20:01:50.299684 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.299662 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-shj6l_openshift-console-operator(b7930caa-d305-4b26-985f-a89a0abb6b08)\"" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" podUID="b7930caa-d305-4b26-985f-a89a0abb6b08" Jul 07 20:01:50.433047 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.432987 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:50.534240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.534207 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:50.633247 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.633161 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:50.716594 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.716563 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.716775 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.716693 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 20:01:50.716849 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.716795 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:51.716756337 +0000 UTC m=+64.225428906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : configmap "kube-rbac-proxy" not found Jul 07 20:01:50.716849 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:50.716829 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:50.716977 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.716960 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 20:01:50.717055 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:50.717045 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:51.717031711 +0000 UTC m=+64.225704279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : secret "insights-runtime-extractor-tls" not found Jul 07 20:01:51.245915 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:51.245879 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:51.346494 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:51.346445 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:51.725264 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:51.725188 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:51.725405 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:51.725299 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:51.725405 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:51.725344 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 20:01:51.725471 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:51.725411 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 20:01:51.725471 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:51.725417 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:53.725401301 +0000 UTC m=+66.234073857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : secret "insights-runtime-extractor-tls" not found Jul 07 20:01:51.725471 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:51.725468 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:53.725455942 +0000 UTC m=+66.234128495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : configmap "kube-rbac-proxy" not found Jul 07 20:01:51.845564 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:51.845537 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t7q6z_2a31e2bb-5d62-4b23-a428-9943115b4cd3/dns-node-resolver/0.log" Jul 07 20:01:52.008783 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.008756 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:52.008949 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.008813 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:01:52.009205 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.009189 2553 scope.go:117] "RemoveContainer" containerID="da09e563e1a2175659f01fea7e15f5f71b5d7120b99619e091a0dd01d4fd57a5" Jul 07 20:01:52.009387 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:52.009366 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-shj6l_openshift-console-operator(b7930caa-d305-4b26-985f-a89a0abb6b08)\"" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" podUID="b7930caa-d305-4b26-985f-a89a0abb6b08" Jul 07 20:01:52.645917 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.645889 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lqwnx_6405eede-1bde-4bf5-b476-ce244e0029eb/node-ca/0.log" Jul 07 20:01:52.736080 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.736042 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:01:52.736234 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:52.736200 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 20:01:52.736274 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:52.736262 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert podName:a400cf5f-3487-46ce-8351-f46672ac0747 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:00.736244576 +0000 UTC m=+73.244917149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-54pwj" (UID: "a400cf5f-3487-46ce-8351-f46672ac0747") : secret "networking-console-plugin-cert" not found Jul 07 20:01:52.856110 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.856080 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xt658"] Jul 07 20:01:52.896904 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.896827 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xt658"] Jul 07 20:01:52.897058 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.896934 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:52.899735 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.899710 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 20:01:52.899856 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.899755 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 20:01:52.899856 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.899789 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2l22k\"" Jul 07 20:01:52.901017 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.900987 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 20:01:52.901203 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.901184 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 20:01:52.925230 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.925200 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:52.937136 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.937107 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8src\" (UniqueName: \"kubernetes.io/projected/5146dba0-02ae-473d-bca1-54db485213cf-kube-api-access-d8src\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:52.937253 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.937217 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5146dba0-02ae-473d-bca1-54db485213cf-signing-key\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:52.937299 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:52.937254 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5146dba0-02ae-473d-bca1-54db485213cf-signing-cabundle\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.018932 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.018897 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:53.037737 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.037712 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5146dba0-02ae-473d-bca1-54db485213cf-signing-cabundle\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.037895 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.037878 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8src\" (UniqueName: \"kubernetes.io/projected/5146dba0-02ae-473d-bca1-54db485213cf-kube-api-access-d8src\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.038046 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.037969 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5146dba0-02ae-473d-bca1-54db485213cf-signing-key\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.038477 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.038459 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5146dba0-02ae-473d-bca1-54db485213cf-signing-cabundle\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.040357 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.040342 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5146dba0-02ae-473d-bca1-54db485213cf-signing-key\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.047433 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.047407 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8src\" (UniqueName: \"kubernetes.io/projected/5146dba0-02ae-473d-bca1-54db485213cf-kube-api-access-d8src\") pod \"service-ca-758f599c5-xt658\" (UID: \"5146dba0-02ae-473d-bca1-54db485213cf\") " pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.121024 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.120974 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:53.206567 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.206499 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xt658" Jul 07 20:01:53.247033 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.247011 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4plnv_b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03/serve-healthcheck-canary/0.log" Jul 07 20:01:53.363271 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.363240 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xt658"] Jul 07 20:01:53.366243 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:53.366214 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5146dba0_02ae_473d_bca1_54db485213cf.slice/crio-338509a461b48f592ec449574cc39797261ec049afc1511d9d2c41b48bccc6e7 WatchSource:0}: Error finding container 338509a461b48f592ec449574cc39797261ec049afc1511d9d2c41b48bccc6e7: Status 404 returned error can't find the container with id 338509a461b48f592ec449574cc39797261ec049afc1511d9d2c41b48bccc6e7 Jul 07 20:01:53.744477 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.744443 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:53.744853 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.744556 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:53.744853 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:53.744597 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 20:01:53.744853 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:53.744649 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 20:01:53.744853 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:53.744687 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:57.74466721 +0000 UTC m=+70.253339763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : secret "insights-runtime-extractor-tls" not found Jul 07 20:01:53.744853 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:53.744703 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:01:57.744696495 +0000 UTC m=+70.253369048 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : configmap "kube-rbac-proxy" not found Jul 07 20:01:53.845487 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.845452 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:53.848776 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.848756 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 20:01:53.858512 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.858486 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e356e3c-7634-4b62-b81d-46cb0a409d97-original-pull-secret\") pod \"global-pull-secret-syncer-dsxqc\" (UID: \"6e356e3c-7634-4b62-b81d-46cb0a409d97\") " pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:53.889351 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.889322 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:53.936538 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.936511 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsxqc" Jul 07 20:01:53.946536 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.946516 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:01:53.949663 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:53.949644 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 20:01:53.957459 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:53.957439 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 20:01:53.957552 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:53.957502 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs podName:d4d270e6-e127-4b3e-a2c0-a7543f1b09fa nodeName:}" failed. No retries permitted until 2026-07-07 20:02:57.957486017 +0000 UTC m=+130.466158571 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs") pod "network-metrics-daemon-dt99m" (UID: "d4d270e6-e127-4b3e-a2c0-a7543f1b09fa") : secret "metrics-daemon-secret" not found Jul 07 20:01:54.047718 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.047685 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:54.050216 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.050196 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tdjg\" (UniqueName: \"kubernetes.io/projected/86bcbc7a-039d-4f27-b439-6b6aad64f748-kube-api-access-4tdjg\") pod \"network-check-target-47mc5\" (UID: \"86bcbc7a-039d-4f27-b439-6b6aad64f748\") " pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:54.070218 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.070194 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dsxqc"] Jul 07 20:01:54.073394 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:54.073370 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e356e3c_7634_4b62_b81d_46cb0a409d97.slice/crio-059eae7c0d80bca5da2640814cee1f9ba465b1b29d8201350c19d42c89529d32 WatchSource:0}: Error finding container 059eae7c0d80bca5da2640814cee1f9ba465b1b29d8201350c19d42c89529d32: Status 404 returned error can't find the container with id 059eae7c0d80bca5da2640814cee1f9ba465b1b29d8201350c19d42c89529d32 Jul 07 20:01:54.315448 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.315361 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xt658" event={"ID":"5146dba0-02ae-473d-bca1-54db485213cf","Type":"ContainerStarted","Data":"0eee5a8d06e3425284bf7a5ac178b3fe3a17389b21daf0787b4a8b1f68c256f5"} Jul 07 20:01:54.315448 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.315414 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xt658" event={"ID":"5146dba0-02ae-473d-bca1-54db485213cf","Type":"ContainerStarted","Data":"338509a461b48f592ec449574cc39797261ec049afc1511d9d2c41b48bccc6e7"} Jul 07 20:01:54.316372 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.316347 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dsxqc" event={"ID":"6e356e3c-7634-4b62-b81d-46cb0a409d97","Type":"ContainerStarted","Data":"059eae7c0d80bca5da2640814cee1f9ba465b1b29d8201350c19d42c89529d32"} Jul 07 20:01:54.335583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.335558 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lcjbq\"" Jul 07 20:01:54.342182 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.342161 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:54.469947 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.469883 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xt658" podStartSLOduration=2.4698608269999998 podStartE2EDuration="2.469860827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:54.343007097 +0000 UTC m=+66.851679670" watchObservedRunningTime="2026-07-07 20:01:54.469860827 +0000 UTC m=+66.978533402" Jul 07 20:01:54.471465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.471326 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47mc5"] Jul 07 20:01:54.474545 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:01:54.474519 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86bcbc7a_039d_4f27_b439_6b6aad64f748.slice/crio-0f4b0f0a8b0230a30ffe650da3ab2185d380c64b2884ce0c5436c3a8a1157145 WatchSource:0}: Error finding container 0f4b0f0a8b0230a30ffe650da3ab2185d380c64b2884ce0c5436c3a8a1157145: Status 404 returned error can't find the container with id 0f4b0f0a8b0230a30ffe650da3ab2185d380c64b2884ce0c5436c3a8a1157145 Jul 07 20:01:54.630427 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.630325 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:54.755575 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.755540 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:01:54.756038 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:54.755669 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 20:01:54.756038 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:54.755734 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls podName:7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b nodeName:}" failed. No retries permitted until 2026-07-07 20:02:26.755715665 +0000 UTC m=+99.264388233 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls") pod "dns-default-9m5fm" (UID: "7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b") : secret "dns-default-metrics-tls" not found Jul 07 20:01:54.756162 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.756132 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:01:54.756261 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:54.756238 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 20:01:54.756313 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:54.756263 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bc8fb5d-76pcm: secret "image-registry-tls" not found Jul 07 20:01:54.756313 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:54.756305 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls podName:fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:26.756293738 +0000 UTC m=+99.264966291 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls") pod "image-registry-559bc8fb5d-76pcm" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92") : secret "image-registry-tls" not found Jul 07 20:01:54.816121 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.816089 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:54.920876 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:54.920780 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:55.022101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:55.022068 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:55.321173 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:55.321137 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47mc5" event={"ID":"86bcbc7a-039d-4f27-b439-6b6aad64f748","Type":"ContainerStarted","Data":"e05f9f51624ac62c9c58da6ca2036a953257bf89c60c66133a5709658d312756"} Jul 07 20:01:55.321369 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:55.321212 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47mc5" event={"ID":"86bcbc7a-039d-4f27-b439-6b6aad64f748","Type":"ContainerStarted","Data":"0f4b0f0a8b0230a30ffe650da3ab2185d380c64b2884ce0c5436c3a8a1157145"} Jul 07 20:01:55.343562 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:55.343517 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:55.343798 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:55.343764 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-47mc5" podStartSLOduration=67.343750959 podStartE2EDuration="1m7.343750959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:01:55.341891551 +0000 UTC m=+67.850564128" watchObservedRunningTime="2026-07-07 20:01:55.343750959 +0000 UTC m=+67.852423533" Jul 07 20:01:56.324916 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:56.324889 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:01:56.455780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:56.455743 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:56.558649 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:56.558614 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:56.734971 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:56.734932 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:56.941508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:56.941477 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:57.143195 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.143068 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:57.336087 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.336054 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:57.480172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.480134 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:57.483093 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.483062 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37c9acc-aeb3-4b6e-8e78-209c1d8227bc-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-wcx8p\" (UID: \"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:57.682606 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.682571 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:57.682779 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.682632 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:01:57.682779 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.682743 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:13.68272604 +0000 UTC m=+86.191398613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : configmap references non-existent config key: service-ca.crt Jul 07 20:01:57.682779 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.682776 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:01:57.682952 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.682793 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:57.682952 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.682848 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 20:01:57.682952 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.682857 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls podName:beb9ec18-4e3a-4655-a6db-6313d35fca26 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:13.682844064 +0000 UTC m=+86.191516621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x9f9x" (UID: "beb9ec18-4e3a-4655-a6db-6313d35fca26") : secret "cluster-monitoring-operator-tls" not found Jul 07 20:01:57.682952 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.682876 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs podName:a73237eb-e751-46a4-ad48-17242e2dd132 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:13.682868643 +0000 UTC m=+86.191541196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs") pod "router-default-757c6f7d89-5mjdr" (UID: "a73237eb-e751-46a4-ad48-17242e2dd132") : secret "router-metrics-certs-default" not found Jul 07 20:01:57.778626 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.778545 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-njb64\"" Jul 07 20:01:57.783594 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.783563 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:57.783737 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.783715 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 20:01:57.783792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.783769 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:01:57.783841 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.783795 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:05.783776746 +0000 UTC m=+78.292449310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : configmap "kube-rbac-proxy" not found Jul 07 20:01:57.783898 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.783852 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 20:01:57.783943 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:01:57.783909 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls podName:08eeb706-817f-4562-859c-f42a542511a3 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:05.783895958 +0000 UTC m=+78.292568525 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s6npk" (UID: "08eeb706-817f-4562-859c-f42a542511a3") : secret "insights-runtime-extractor-tls" not found Jul 07 20:01:57.786417 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.786401 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" Jul 07 20:01:57.960065 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:57.960037 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p"] Jul 07 20:01:58.076798 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:58.076705 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:58.331568 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:58.331490 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" event={"ID":"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc","Type":"ContainerStarted","Data":"917dab50f20830737d94ce8ea8a12870240ce9c4358a1f4ca19d56b0291caa49"} Jul 07 20:01:58.334310 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:58.334287 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:01:59.523440 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:01:59.523269 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:00.753794 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:00.753755 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:00.811451 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:00.811413 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:02:00.811636 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:02:00.811584 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 20:02:00.811701 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:02:00.811660 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert podName:a400cf5f-3487-46ce-8351-f46672ac0747 nodeName:}" failed. No retries permitted until 2026-07-07 20:02:16.811639825 +0000 UTC m=+89.320312377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-54pwj" (UID: "a400cf5f-3487-46ce-8351-f46672ac0747") : secret "networking-console-plugin-cert" not found Jul 07 20:02:00.851090 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:00.851034 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:00.955665 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:00.955629 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:01.058411 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:01.058315 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:01.155591 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:01.155555 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:01.774300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:01.774230 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:02.151625 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:02.151579 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:02.347913 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:02.347820 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dsxqc" event={"ID":"6e356e3c-7634-4b62-b81d-46cb0a409d97","Type":"ContainerStarted","Data":"1c307d787a701484d2f515532cb555b2e7610cb126149cb183881f4f7dd1eedf"} Jul 07 20:02:02.349722 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:02.349682 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" event={"ID":"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc","Type":"ContainerStarted","Data":"327086f59e9f60228dd7ecafe2f6a54e6ed31f8e80eac2ab2b5bc37a477aaa2b"} Jul 07 20:02:02.349831 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:02.349735 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" event={"ID":"b37c9acc-aeb3-4b6e-8e78-209c1d8227bc","Type":"ContainerStarted","Data":"942a76dcdb967cdef26307a10bb397ce52d13ed5a3d47bec3d2e8a4e54446895"} Jul 07 20:02:02.371553 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:02.371505 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dsxqc" podStartSLOduration=66.498496829 podStartE2EDuration="1m14.371488188s" totalImagesPullingTime="7.872991359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:02.370520361 +0000 UTC m=+74.879192936" watchObservedRunningTime="2026-07-07 20:02:02.371488188 +0000 UTC m=+74.880160764" Jul 07 20:02:02.394803 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:02.394759 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-wcx8p" podStartSLOduration=17.464024886 podStartE2EDuration="21.394745609s" totalImagesPullingTime="3.930720723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:02.393414086 +0000 UTC m=+74.902086660" watchObservedRunningTime="2026-07-07 20:02:02.394745609 +0000 UTC m=+74.903418185" Jul 07 20:02:02.539442 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:02.539399 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:03.801331 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:03.801235 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:03.909269 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:03.909233 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:04.153257 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:04.153172 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:04.352722 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:04.352685 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:04.750241 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:04.750203 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:05.143171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.143096 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:05.338210 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.338178 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:05.826138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.826109 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:05.858384 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.858353 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:02:05.858499 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.858445 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:02:05.858965 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.858945 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08eeb706-817f-4562-859c-f42a542511a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:02:05.860570 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.860544 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08eeb706-817f-4562-859c-f42a542511a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s6npk\" (UID: \"08eeb706-817f-4562-859c-f42a542511a3\") " pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:02:05.958060 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:05.958028 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s6npk" Jul 07 20:02:06.116105 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:06.116079 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s6npk"] Jul 07 20:02:06.118376 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:06.118342 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08eeb706_817f_4562_859c_f42a542511a3.slice/crio-a114d99bc2c73d1ce7c685732490b7c28abc7e8e79c26547dec80bc0aaeb2a11 WatchSource:0}: Error finding container a114d99bc2c73d1ce7c685732490b7c28abc7e8e79c26547dec80bc0aaeb2a11: Status 404 returned error can't find the container with id a114d99bc2c73d1ce7c685732490b7c28abc7e8e79c26547dec80bc0aaeb2a11 Jul 07 20:02:06.362642 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:06.362608 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s6npk" event={"ID":"08eeb706-817f-4562-859c-f42a542511a3","Type":"ContainerStarted","Data":"a114d99bc2c73d1ce7c685732490b7c28abc7e8e79c26547dec80bc0aaeb2a11"} Jul 07 20:02:07.023871 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:07.023805 2553 scope.go:117] "RemoveContainer" containerID="da09e563e1a2175659f01fea7e15f5f71b5d7120b99619e091a0dd01d4fd57a5" Jul 07 20:02:07.368815 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:07.368689 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:02:07.369261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:07.369162 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/1.log" Jul 07 20:02:07.369261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:07.369191 2553 generic.go:356] "Generic (PLEG): container finished" podID="b7930caa-d305-4b26-985f-a89a0abb6b08" containerID="84af669fa4fbbc7c3e64140da12805aae1cd91cdb0b9b28b52f32f4bdd423f82" exitCode=255 Jul 07 20:02:07.369261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:07.369237 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" event={"ID":"b7930caa-d305-4b26-985f-a89a0abb6b08","Type":"ContainerDied","Data":"84af669fa4fbbc7c3e64140da12805aae1cd91cdb0b9b28b52f32f4bdd423f82"} Jul 07 20:02:07.369435 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:07.369291 2553 scope.go:117] "RemoveContainer" containerID="da09e563e1a2175659f01fea7e15f5f71b5d7120b99619e091a0dd01d4fd57a5" Jul 07 20:02:07.369748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:07.369720 2553 scope.go:117] "RemoveContainer" containerID="84af669fa4fbbc7c3e64140da12805aae1cd91cdb0b9b28b52f32f4bdd423f82" Jul 07 20:02:07.369957 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:02:07.369930 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-shj6l_openshift-console-operator(b7930caa-d305-4b26-985f-a89a0abb6b08)\"" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" podUID="b7930caa-d305-4b26-985f-a89a0abb6b08" Jul 07 20:02:08.059668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:08.059633 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:08.141314 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:08.141279 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:08.378481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:08.378405 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:02:10.387693 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:10.387620 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s6npk" event={"ID":"08eeb706-817f-4562-859c-f42a542511a3","Type":"ContainerStarted","Data":"221e742345fe91eeded14bce23847a559ae6e4415a8a7037c3d415d2d207b6c8"} Jul 07 20:02:11.392793 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:11.392758 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s6npk" event={"ID":"08eeb706-817f-4562-859c-f42a542511a3","Type":"ContainerStarted","Data":"74a56944f88db9a106ce7baa1797c0dac6f1104ab3157e7c0f99fc0dba008b04"} Jul 07 20:02:12.008681 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:12.008641 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:02:12.008864 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:12.008701 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:02:12.009042 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:12.009028 2553 scope.go:117] "RemoveContainer" containerID="84af669fa4fbbc7c3e64140da12805aae1cd91cdb0b9b28b52f32f4bdd423f82" Jul 07 20:02:12.009217 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:02:12.009195 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-shj6l_openshift-console-operator(b7930caa-d305-4b26-985f-a89a0abb6b08)\"" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" podUID="b7930caa-d305-4b26-985f-a89a0abb6b08" Jul 07 20:02:13.402209 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.402170 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s6npk" event={"ID":"08eeb706-817f-4562-859c-f42a542511a3","Type":"ContainerStarted","Data":"917e7d98fdd18732ee7cad61531076b47171e647a6e65cde20d498fa2c9d20bc"} Jul 07 20:02:13.428722 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.428675 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s6npk" podStartSLOduration=16.367739473 podStartE2EDuration="23.428654318s" totalImagesPullingTime="7.060914845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:01:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:13.427411094 +0000 UTC m=+85.936083668" watchObservedRunningTime="2026-07-07 20:02:13.428654318 +0000 UTC m=+85.937326893" Jul 07 20:02:13.729980 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.729953 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:02:13.730126 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.730071 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:13.730126 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.730099 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:13.730718 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.730696 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a73237eb-e751-46a4-ad48-17242e2dd132-service-ca-bundle\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:13.732245 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.732216 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/beb9ec18-4e3a-4655-a6db-6313d35fca26-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x9f9x\" (UID: \"beb9ec18-4e3a-4655-a6db-6313d35fca26\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:02:13.732347 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.732303 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73237eb-e751-46a4-ad48-17242e2dd132-metrics-certs\") pod \"router-default-757c6f7d89-5mjdr\" (UID: \"a73237eb-e751-46a4-ad48-17242e2dd132\") " pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:13.951479 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.951445 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h82kq\"" Jul 07 20:02:13.958717 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.958697 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" Jul 07 20:02:13.962538 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.962520 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j6m9q\"" Jul 07 20:02:13.970301 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:13.970281 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:14.091737 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.091707 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x"] Jul 07 20:02:14.094626 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:14.094597 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb9ec18_4e3a_4655_a6db_6313d35fca26.slice/crio-8bc9abcabb58c516ec20828e7b2085272a53e1f254cf1045d92bfb57159ef6ae WatchSource:0}: Error finding container 8bc9abcabb58c516ec20828e7b2085272a53e1f254cf1045d92bfb57159ef6ae: Status 404 returned error can't find the container with id 8bc9abcabb58c516ec20828e7b2085272a53e1f254cf1045d92bfb57159ef6ae Jul 07 20:02:14.111243 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.111216 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-757c6f7d89-5mjdr"] Jul 07 20:02:14.115275 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:14.115250 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73237eb_e751_46a4_ad48_17242e2dd132.slice/crio-0611a062edd9a1033160669c013d23728945e01482b3fa0ba79e1a2f14dfef58 WatchSource:0}: Error finding container 0611a062edd9a1033160669c013d23728945e01482b3fa0ba79e1a2f14dfef58: Status 404 returned error can't find the container with id 0611a062edd9a1033160669c013d23728945e01482b3fa0ba79e1a2f14dfef58 Jul 07 20:02:14.409810 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.409725 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" event={"ID":"a73237eb-e751-46a4-ad48-17242e2dd132","Type":"ContainerStarted","Data":"06bf4e71c2d9c62796b153d85a529df877c22b86ba072de58b419d8b7d629c3d"} Jul 07 20:02:14.409810 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.409780 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" event={"ID":"a73237eb-e751-46a4-ad48-17242e2dd132","Type":"ContainerStarted","Data":"0611a062edd9a1033160669c013d23728945e01482b3fa0ba79e1a2f14dfef58"} Jul 07 20:02:14.410869 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.410846 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" event={"ID":"beb9ec18-4e3a-4655-a6db-6313d35fca26","Type":"ContainerStarted","Data":"8bc9abcabb58c516ec20828e7b2085272a53e1f254cf1045d92bfb57159ef6ae"} Jul 07 20:02:14.441217 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.441171 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" podStartSLOduration=33.441158233 podStartE2EDuration="33.441158233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:14.438520902 +0000 UTC m=+86.947193478" watchObservedRunningTime="2026-07-07 20:02:14.441158233 +0000 UTC m=+86.949830808" Jul 07 20:02:14.971315 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.971278 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:14.974537 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:14.974511 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:15.414435 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:15.414378 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:15.415660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:15.415637 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-757c6f7d89-5mjdr" Jul 07 20:02:16.418912 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.418876 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" event={"ID":"beb9ec18-4e3a-4655-a6db-6313d35fca26","Type":"ContainerStarted","Data":"9017c475d13c38c2e1a7034df907adbde47a8048c88a6ce2789a0fe29e3332df"} Jul 07 20:02:16.445677 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.445616 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x9f9x" podStartSLOduration=33.234202541 podStartE2EDuration="35.445567836s" totalImagesPullingTime="2.211365295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:16.44447005 +0000 UTC m=+88.953142626" watchObservedRunningTime="2026-07-07 20:02:16.445567836 +0000 UTC m=+88.954240413" Jul 07 20:02:16.716838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.716753 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:16.830384 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.830331 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:16.860244 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.860215 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:02:16.862596 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.862556 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a400cf5f-3487-46ce-8351-f46672ac0747-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-54pwj\" (UID: \"a400cf5f-3487-46ce-8351-f46672ac0747\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:02:16.942836 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.942807 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:16.995796 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:16.995771 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mhhnd\"" Jul 07 20:02:17.002503 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:17.002481 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" Jul 07 20:02:17.140742 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:17.140716 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-54pwj"] Jul 07 20:02:17.142057 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:17.142026 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda400cf5f_3487_46ce_8351_f46672ac0747.slice/crio-d233cd25ca715a6b40613f4ee3aad7c699974b113e903513e5b936eec144cc7a WatchSource:0}: Error finding container d233cd25ca715a6b40613f4ee3aad7c699974b113e903513e5b936eec144cc7a: Status 404 returned error can't find the container with id d233cd25ca715a6b40613f4ee3aad7c699974b113e903513e5b936eec144cc7a Jul 07 20:02:17.423620 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:17.423543 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" event={"ID":"a400cf5f-3487-46ce-8351-f46672ac0747","Type":"ContainerStarted","Data":"d233cd25ca715a6b40613f4ee3aad7c699974b113e903513e5b936eec144cc7a"} Jul 07 20:02:17.680176 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:17.680075 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:18.054658 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.054629 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk"] Jul 07 20:02:18.089421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.089393 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk"] Jul 07 20:02:18.089576 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.089503 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" Jul 07 20:02:18.094276 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.094251 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 20:02:18.095548 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.095529 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-nfmxb\"" Jul 07 20:02:18.200283 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.200205 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:18.269300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.269265 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c1e984ed-a4bc-4a76-902d-8e623fb91e42-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x98qk\" (UID: \"c1e984ed-a4bc-4a76-902d-8e623fb91e42\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" Jul 07 20:02:18.324220 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.324188 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:18.370171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.370140 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c1e984ed-a4bc-4a76-902d-8e623fb91e42-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x98qk\" (UID: \"c1e984ed-a4bc-4a76-902d-8e623fb91e42\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" Jul 07 20:02:18.372354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.372332 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c1e984ed-a4bc-4a76-902d-8e623fb91e42-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x98qk\" (UID: \"c1e984ed-a4bc-4a76-902d-8e623fb91e42\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" Jul 07 20:02:18.399415 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.399379 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" Jul 07 20:02:18.529499 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.529398 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:18.708674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.708643 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:18.824528 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.824500 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:18.860780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:18.860753 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk"] Jul 07 20:02:18.864762 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:18.864723 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e984ed_a4bc_4a76_902d_8e623fb91e42.slice/crio-94f0ad421724c0886ce51b51da46962be1d28ea94045bd52068d76edebad65f2 WatchSource:0}: Error finding container 94f0ad421724c0886ce51b51da46962be1d28ea94045bd52068d76edebad65f2: Status 404 returned error can't find the container with id 94f0ad421724c0886ce51b51da46962be1d28ea94045bd52068d76edebad65f2 Jul 07 20:02:19.059612 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:19.059535 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:19.173134 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:19.173109 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:19.432107 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:19.432031 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" event={"ID":"a400cf5f-3487-46ce-8351-f46672ac0747","Type":"ContainerStarted","Data":"d1dfd34789cbf790d3ad6853731eef99d0af374c39cecd2ae43c09b4d7f3c453"} Jul 07 20:02:19.433233 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:19.433210 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" event={"ID":"c1e984ed-a4bc-4a76-902d-8e623fb91e42","Type":"ContainerStarted","Data":"94f0ad421724c0886ce51b51da46962be1d28ea94045bd52068d76edebad65f2"} Jul 07 20:02:19.462974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:19.462927 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-54pwj" podStartSLOduration=33.510247586 podStartE2EDuration="35.462914293s" totalImagesPullingTime="1.952666707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:19.462544145 +0000 UTC m=+91.971216721" watchObservedRunningTime="2026-07-07 20:02:19.462914293 +0000 UTC m=+91.971586868" Jul 07 20:02:19.642982 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:19.642946 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:19.760786 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:19.760755 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:21.441199 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:21.441157 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" event={"ID":"c1e984ed-a4bc-4a76-902d-8e623fb91e42","Type":"ContainerStarted","Data":"abf09e78be6c91fb1ee7645afad559f733b4499fa94c658396f1dd02c5556afa"} Jul 07 20:02:21.441584 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:21.441322 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" Jul 07 20:02:21.445919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:21.445894 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" Jul 07 20:02:21.465258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:21.465211 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x98qk" podStartSLOduration=2.5254411169999997 podStartE2EDuration="4.465196005s" totalImagesPullingTime="1.939754888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:21.464801652 +0000 UTC m=+93.973474231" watchObservedRunningTime="2026-07-07 20:02:21.465196005 +0000 UTC m=+93.973868581" Jul 07 20:02:21.947935 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:21.947900 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:22.081301 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:22.081264 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:22.538310 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:22.538278 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:23.062570 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:23.062529 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:23.253860 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:23.253823 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:25.057342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:25.057304 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:25.153785 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:25.153749 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:25.222540 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:25.222509 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:25.369525 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:25.369450 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:25.468647 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:25.468618 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:25.570870 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:25.570835 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:25.650984 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:25.650913 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:26.736736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:26.736699 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:26.836107 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:26.836069 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:02:26.836262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:26.836110 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:02:26.838535 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:26.838511 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b-metrics-tls\") pod \"dns-default-9m5fm\" (UID: \"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b\") " pod="openshift-dns/dns-default-9m5fm" Jul 07 20:02:26.838633 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:26.838511 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"image-registry-559bc8fb5d-76pcm\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:02:27.024250 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.024167 2553 scope.go:117] "RemoveContainer" containerID="84af669fa4fbbc7c3e64140da12805aae1cd91cdb0b9b28b52f32f4bdd423f82" Jul 07 20:02:27.024408 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:02:27.024381 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-shj6l_openshift-console-operator(b7930caa-d305-4b26-985f-a89a0abb6b08)\"" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" podUID="b7930caa-d305-4b26-985f-a89a0abb6b08" Jul 07 20:02:27.062024 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.061982 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fmqz4\"" Jul 07 20:02:27.069529 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.069511 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:02:27.070383 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.070368 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s6xh9\"" Jul 07 20:02:27.077641 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.077617 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9m5fm" Jul 07 20:02:27.216744 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.216705 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559bc8fb5d-76pcm"] Jul 07 20:02:27.220221 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:27.220195 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0c9f14_5763_4e1a_b904_9cdbeb8e4b92.slice/crio-d583aeafa2030eb7a924daa893c0a593da1eded78ed1585d67900d1c78667b05 WatchSource:0}: Error finding container d583aeafa2030eb7a924daa893c0a593da1eded78ed1585d67900d1c78667b05: Status 404 returned error can't find the container with id d583aeafa2030eb7a924daa893c0a593da1eded78ed1585d67900d1c78667b05 Jul 07 20:02:27.225556 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.225534 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9m5fm"] Jul 07 20:02:27.228598 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:27.228572 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0c8a7b_4eec_40e4_a2bc_563cea6d5a4b.slice/crio-e1208890062ac54a2e8117f9add2a252536cb221759b32d5af695cc70197539e WatchSource:0}: Error finding container e1208890062ac54a2e8117f9add2a252536cb221759b32d5af695cc70197539e: Status 404 returned error can't find the container with id e1208890062ac54a2e8117f9add2a252536cb221759b32d5af695cc70197539e Jul 07 20:02:27.330070 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.330018 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-47mc5" Jul 07 20:02:27.465916 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.465877 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m5fm" event={"ID":"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b","Type":"ContainerStarted","Data":"e1208890062ac54a2e8117f9add2a252536cb221759b32d5af695cc70197539e"} Jul 07 20:02:27.467423 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.467390 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" event={"ID":"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92","Type":"ContainerStarted","Data":"bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800"} Jul 07 20:02:27.467578 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.467440 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" event={"ID":"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92","Type":"ContainerStarted","Data":"d583aeafa2030eb7a924daa893c0a593da1eded78ed1585d67900d1c78667b05"} Jul 07 20:02:27.467578 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.467529 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:02:27.496847 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:27.496795 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" podStartSLOduration=99.496779054 podStartE2EDuration="1m39.496779054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:27.49589492 +0000 UTC m=+100.004567495" watchObservedRunningTime="2026-07-07 20:02:27.496779054 +0000 UTC m=+100.005451628" Jul 07 20:02:29.156486 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.156457 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:29.384393 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.384306 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:29.480682 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.480649 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m5fm" event={"ID":"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b","Type":"ContainerStarted","Data":"be2da65c182a00feb1ce9e1b85ea00904b7c4c9f4c4829a40cf3d8080a208d59"} Jul 07 20:02:29.480868 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.480701 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9m5fm" event={"ID":"7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b","Type":"ContainerStarted","Data":"74a9ddeb2a4c3eddc9e15987b1468ceca461b29de6387a3578b3dbde86bfba06"} Jul 07 20:02:29.480868 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.480753 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9m5fm" Jul 07 20:02:29.497340 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.497311 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:29.498198 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.498173 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fttmr"] Jul 07 20:02:29.501795 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.501776 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.505155 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.505123 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 20:02:29.505398 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.505379 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 20:02:29.505535 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.505516 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-24vrt\"" Jul 07 20:02:29.506756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.506587 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 20:02:29.506756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.506653 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 20:02:29.508096 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.508046 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9m5fm" podStartSLOduration=65.671987705 podStartE2EDuration="1m7.508027429s" totalImagesPullingTime="1.836039724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:29.506649997 +0000 UTC m=+102.015322575" watchObservedRunningTime="2026-07-07 20:02:29.508027429 +0000 UTC m=+102.016699999" Jul 07 20:02:29.554469 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554430 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2ql5\" (UniqueName: \"kubernetes.io/projected/9894d7fb-7f85-4675-97af-639fd731ba0a-kube-api-access-m2ql5\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554631 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554496 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-tls\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554631 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554577 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554729 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554628 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-sys\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554729 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554651 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9894d7fb-7f85-4675-97af-639fd731ba0a-metrics-client-ca\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554729 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554711 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554847 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554773 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-root\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554847 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554803 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-textfile\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.554847 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.554823 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-wtmp\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.597794 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.597757 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:29.655748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655664 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-root\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.655748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655702 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-textfile\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.655748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655720 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-wtmp\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.655748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655737 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2ql5\" (UniqueName: \"kubernetes.io/projected/9894d7fb-7f85-4675-97af-639fd731ba0a-kube-api-access-m2ql5\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655764 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-tls\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655804 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-root\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655894 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-wtmp\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.655817 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.656028 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-sys\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.656057 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9894d7fb-7f85-4675-97af-639fd731ba0a-metrics-client-ca\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656369 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.656089 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-textfile\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656369 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.656097 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656369 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.656184 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9894d7fb-7f85-4675-97af-639fd731ba0a-sys\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656658 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.656623 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9894d7fb-7f85-4675-97af-639fd731ba0a-metrics-client-ca\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.656658 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.656623 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.658355 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.658334 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-tls\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.658492 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.658474 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9894d7fb-7f85-4675-97af-639fd731ba0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.667566 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.667542 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2ql5\" (UniqueName: \"kubernetes.io/projected/9894d7fb-7f85-4675-97af-639fd731ba0a-kube-api-access-m2ql5\") pod \"node-exporter-fttmr\" (UID: \"9894d7fb-7f85-4675-97af-639fd731ba0a\") " pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.697660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.697622 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:29.813411 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.813377 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fttmr" Jul 07 20:02:29.825212 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:29.825186 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9894d7fb_7f85_4675_97af_639fd731ba0a.slice/crio-eaf61336a1211a3866d37cf06e877bfa989b0d08606cbc20eb4c851e39f9734c WatchSource:0}: Error finding container eaf61336a1211a3866d37cf06e877bfa989b0d08606cbc20eb4c851e39f9734c: Status 404 returned error can't find the container with id eaf61336a1211a3866d37cf06e877bfa989b0d08606cbc20eb4c851e39f9734c Jul 07 20:02:29.862192 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:29.862151 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:30.486635 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:30.486591 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fttmr" event={"ID":"9894d7fb-7f85-4675-97af-639fd731ba0a","Type":"ContainerStarted","Data":"eaf61336a1211a3866d37cf06e877bfa989b0d08606cbc20eb4c851e39f9734c"} Jul 07 20:02:31.892116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:31.892074 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:31.971306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:31.971257 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:32.453255 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:32.453225 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:32.495983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:32.495955 2553 generic.go:356] "Generic (PLEG): container finished" podID="9894d7fb-7f85-4675-97af-639fd731ba0a" containerID="94e2501e71ffd3a5d2b579209406b3556e83180127ddf8b28130388638f04e6d" exitCode=0 Jul 07 20:02:32.496101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:32.496016 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fttmr" event={"ID":"9894d7fb-7f85-4675-97af-639fd731ba0a","Type":"ContainerDied","Data":"94e2501e71ffd3a5d2b579209406b3556e83180127ddf8b28130388638f04e6d"} Jul 07 20:02:32.576775 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:32.576752 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:32.687932 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:32.687895 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:32.861317 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:32.861237 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:33.502334 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:33.502300 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fttmr" event={"ID":"9894d7fb-7f85-4675-97af-639fd731ba0a","Type":"ContainerStarted","Data":"c9b35e1f7023818609f7afea963203b6db72d1ed9ad5f9efd40e499de1b10f56"} Jul 07 20:02:33.502747 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:33.502356 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fttmr" event={"ID":"9894d7fb-7f85-4675-97af-639fd731ba0a","Type":"ContainerStarted","Data":"60edc0ef08a238a5755f7915549bcf1de6b3766f98e8ba58038225079f677bc3"} Jul 07 20:02:33.534903 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:33.534853 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fttmr" podStartSLOduration=1.958668582 podStartE2EDuration="4.534836826s" totalImagesPullingTime="2.509711093s" totalInitContainerRuntime="66.457151ms" isStatefulPod=true podCreationTimestamp="2026-07-07 20:02:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:33.532409934 +0000 UTC m=+106.041082519" watchObservedRunningTime="2026-07-07 20:02:33.534836826 +0000 UTC m=+106.043509401" Jul 07 20:02:33.664765 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:33.664729 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:33.778053 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:33.777954 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:34.153770 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:34.153692 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:34.239213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:34.239175 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:34.474055 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:34.473951 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:34.569335 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:34.569303 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:35.810866 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:35.810835 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:38.025387 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.025355 2553 scope.go:117] "RemoveContainer" containerID="84af669fa4fbbc7c3e64140da12805aae1cd91cdb0b9b28b52f32f4bdd423f82" Jul 07 20:02:38.521396 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.521369 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:02:38.521556 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.521477 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" event={"ID":"b7930caa-d305-4b26-985f-a89a0abb6b08","Type":"ContainerStarted","Data":"9832fcc9741823ebd2e55cf2aef46679b347753e973126ce1dad74b916d29e32"} Jul 07 20:02:38.521762 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.521747 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:02:38.543777 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.543734 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" podStartSLOduration=51.686795199 podStartE2EDuration="57.543720911s" totalImagesPullingTime="5.856925712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:38.542248696 +0000 UTC m=+111.050921273" watchObservedRunningTime="2026-07-07 20:02:38.543720911 +0000 UTC m=+111.052393486" Jul 07 20:02:38.604031 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.603976 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-shj6l" Jul 07 20:02:38.834345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.834269 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-pt5cw"] Jul 07 20:02:38.837011 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.836973 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-pt5cw" Jul 07 20:02:38.841887 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.841858 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 20:02:38.845495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.844058 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:38.845495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.844781 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-s2hjd\"" Jul 07 20:02:38.845495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.845101 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 20:02:38.853930 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.853910 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-pt5cw"] Jul 07 20:02:38.936048 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.935986 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qn2d\" (UniqueName: \"kubernetes.io/projected/6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41-kube-api-access-7qn2d\") pod \"downloads-685bfb5f64-pt5cw\" (UID: \"6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41\") " pod="openshift-console/downloads-685bfb5f64-pt5cw" Jul 07 20:02:38.940831 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:38.940811 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:39.037074 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:39.037042 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qn2d\" (UniqueName: \"kubernetes.io/projected/6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41-kube-api-access-7qn2d\") pod \"downloads-685bfb5f64-pt5cw\" (UID: \"6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41\") " pod="openshift-console/downloads-685bfb5f64-pt5cw" Jul 07 20:02:39.057887 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:39.057856 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qn2d\" (UniqueName: \"kubernetes.io/projected/6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41-kube-api-access-7qn2d\") pod \"downloads-685bfb5f64-pt5cw\" (UID: \"6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41\") " pod="openshift-console/downloads-685bfb5f64-pt5cw" Jul 07 20:02:39.149269 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:39.149199 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-pt5cw" Jul 07 20:02:39.283247 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:39.283214 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-pt5cw"] Jul 07 20:02:39.286745 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:39.286708 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6631ff86_3ee8_44d9_8ef5_ba5e2bcd4d41.slice/crio-e6192eaf36eb71cde5e0c234f3a3d6b28921187d978697c8ca7099cace9b7e45 WatchSource:0}: Error finding container e6192eaf36eb71cde5e0c234f3a3d6b28921187d978697c8ca7099cace9b7e45: Status 404 returned error can't find the container with id e6192eaf36eb71cde5e0c234f3a3d6b28921187d978697c8ca7099cace9b7e45 Jul 07 20:02:39.298521 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:39.298499 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:39.489318 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:39.489288 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9m5fm" Jul 07 20:02:39.526397 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:39.526361 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-pt5cw" event={"ID":"6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41","Type":"ContainerStarted","Data":"e6192eaf36eb71cde5e0c234f3a3d6b28921187d978697c8ca7099cace9b7e45"} Jul 07 20:02:41.230516 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:41.230477 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:41.330156 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:41.330118 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:41.722328 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:41.722238 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:41.822734 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:41.822696 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:44.105698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:44.105651 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:44.205844 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:44.205806 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:44.995705 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:44.995676 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559bc8fb5d-76pcm"] Jul 07 20:02:45.001064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:45.001040 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:02:45.129986 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:45.129949 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:45.232317 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:45.232283 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:45.347829 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:45.347714 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:48.754893 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.754857 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dfb898459-v78qp"] Jul 07 20:02:48.765541 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.765510 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.773839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.773801 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 20:02:48.773977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.773909 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 20:02:48.774079 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.773971 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 20:02:48.774315 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.774299 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 20:02:48.774413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.774393 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb898459-v78qp"] Jul 07 20:02:48.775149 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.775132 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v9tv2\"" Jul 07 20:02:48.775251 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.775194 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 20:02:48.823473 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.823451 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-serving-cert\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.823570 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.823498 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-config\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.823570 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.823523 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-oauth-serving-cert\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.823648 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.823566 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmj9n\" (UniqueName: \"kubernetes.io/projected/2c46bf51-9e02-48e4-a238-c5e16a0fa425-kube-api-access-dmj9n\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.823648 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.823589 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-service-ca\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.823712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.823648 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-oauth-config\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925068 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925033 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-oauth-serving-cert\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925076 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmj9n\" (UniqueName: \"kubernetes.io/projected/2c46bf51-9e02-48e4-a238-c5e16a0fa425-kube-api-access-dmj9n\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925099 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-service-ca\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925339 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925283 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-oauth-config\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925367 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-serving-cert\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925451 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-config\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925822 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925784 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-oauth-serving-cert\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.925942 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.925902 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-service-ca\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.926115 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.926091 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-config\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.927886 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.927868 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-serving-cert\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.928008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.927889 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-oauth-config\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:48.939639 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:48.939612 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmj9n\" (UniqueName: \"kubernetes.io/projected/2c46bf51-9e02-48e4-a238-c5e16a0fa425-kube-api-access-dmj9n\") pod \"console-6dfb898459-v78qp\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:49.075602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:49.075514 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:49.229659 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:49.229630 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb898459-v78qp"] Jul 07 20:02:49.235092 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:02:49.235060 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c46bf51_9e02_48e4_a238_c5e16a0fa425.slice/crio-158e6758beca5f07e5220b89dd07a26e004e5e78cb4ef7497cfbd3775c6b62ae WatchSource:0}: Error finding container 158e6758beca5f07e5220b89dd07a26e004e5e78cb4ef7497cfbd3775c6b62ae: Status 404 returned error can't find the container with id 158e6758beca5f07e5220b89dd07a26e004e5e78cb4ef7497cfbd3775c6b62ae Jul 07 20:02:49.566253 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:49.566216 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb898459-v78qp" event={"ID":"2c46bf51-9e02-48e4-a238-c5e16a0fa425","Type":"ContainerStarted","Data":"158e6758beca5f07e5220b89dd07a26e004e5e78cb4ef7497cfbd3775c6b62ae"} Jul 07 20:02:53.071210 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:53.071170 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:53.147600 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:53.147562 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:53.586824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:53.586785 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb898459-v78qp" event={"ID":"2c46bf51-9e02-48e4-a238-c5e16a0fa425","Type":"ContainerStarted","Data":"4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416"} Jul 07 20:02:53.612664 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:53.612619 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dfb898459-v78qp" podStartSLOduration=1.63794816 podStartE2EDuration="5.612603751s" totalImagesPullingTime="3.974655591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:02:53.610620101 +0000 UTC m=+126.119292677" watchObservedRunningTime="2026-07-07 20:02:53.612603751 +0000 UTC m=+126.121276325" Jul 07 20:02:54.120783 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:54.120734 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:54.221905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:54.221863 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:54.592983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:54.592945 2553 generic.go:356] "Generic (PLEG): container finished" podID="d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a" containerID="a7d7ccc97ac65a3f2db8c69b7956535b977389c8dceb700d5022bd7ebed1a7ea" exitCode=0 Jul 07 20:02:54.593172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:54.593031 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" event={"ID":"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a","Type":"ContainerDied","Data":"a7d7ccc97ac65a3f2db8c69b7956535b977389c8dceb700d5022bd7ebed1a7ea"} Jul 07 20:02:54.593792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:54.593503 2553 scope.go:117] "RemoveContainer" containerID="a7d7ccc97ac65a3f2db8c69b7956535b977389c8dceb700d5022bd7ebed1a7ea" Jul 07 20:02:55.598628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:55.598586 2553 generic.go:356] "Generic (PLEG): container finished" podID="1cac9670-2657-43c9-a27a-a12fd7f54db2" containerID="0c48d69728bb04ebeb0c891001c8570a6a21aaa1527ff55f8fa053e1f633e87f" exitCode=0 Jul 07 20:02:55.599104 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:55.598669 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" event={"ID":"1cac9670-2657-43c9-a27a-a12fd7f54db2","Type":"ContainerDied","Data":"0c48d69728bb04ebeb0c891001c8570a6a21aaa1527ff55f8fa053e1f633e87f"} Jul 07 20:02:55.599104 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:55.599079 2553 scope.go:117] "RemoveContainer" containerID="0c48d69728bb04ebeb0c891001c8570a6a21aaa1527ff55f8fa053e1f633e87f" Jul 07 20:02:55.600907 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:55.600880 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-8795t" event={"ID":"d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a","Type":"ContainerStarted","Data":"0fcbb0950c7e73e392a8e06c5e0735726e75d0e8a110f4d6622f5461870705cf"} Jul 07 20:02:56.608274 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:56.608227 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7qwcb" event={"ID":"1cac9670-2657-43c9-a27a-a12fd7f54db2","Type":"ContainerStarted","Data":"4c30071aeb04c13180fd184df1aa2798dc50dc682cbba2de6de4fcf323d7b103"} Jul 07 20:02:57.534386 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.534342 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59bdc756d5-nldrf"] Jul 07 20:02:57.538973 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.538948 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.551461 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.551440 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 20:02:57.552931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.552907 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59bdc756d5-nldrf"] Jul 07 20:02:57.607438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.607405 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lgv\" (UniqueName: \"kubernetes.io/projected/bef38548-70c7-4cbe-9224-8692f7605752-kube-api-access-84lgv\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.607583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.607449 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-service-ca\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.607583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.607469 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-serving-cert\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.607583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.607523 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-trusted-ca-bundle\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.607716 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.607611 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-oauth-config\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.607716 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.607668 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-console-config\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.607716 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.607692 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-oauth-serving-cert\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.708833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.708795 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-console-config\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.708833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.708851 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-oauth-serving-cert\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709406 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.708894 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84lgv\" (UniqueName: \"kubernetes.io/projected/bef38548-70c7-4cbe-9224-8692f7605752-kube-api-access-84lgv\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709406 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.708929 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-service-ca\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709406 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.709273 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-serving-cert\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709406 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.709322 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-trusted-ca-bundle\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709608 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.709409 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-oauth-config\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709679 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.709655 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-console-config\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709775 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.709752 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-service-ca\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.709866 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.709752 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-oauth-serving-cert\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.710402 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.710380 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-trusted-ca-bundle\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.712038 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.712012 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-oauth-config\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.712220 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.712204 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-serving-cert\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.720399 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.720375 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lgv\" (UniqueName: \"kubernetes.io/projected/bef38548-70c7-4cbe-9224-8692f7605752-kube-api-access-84lgv\") pod \"console-59bdc756d5-nldrf\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:57.850174 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:57.850087 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:02:58.012416 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:58.012378 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:02:58.014952 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:58.014922 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4d270e6-e127-4b3e-a2c0-a7543f1b09fa-metrics-certs\") pod \"network-metrics-daemon-dt99m\" (UID: \"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa\") " pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:02:58.144531 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:58.144440 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mcw2j\"" Jul 07 20:02:58.152839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:58.152814 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dt99m" Jul 07 20:02:58.155607 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:58.155472 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:58.253615 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:58.253561 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:02:59.076498 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:59.076460 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:59.076958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:59.076559 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:59.081959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:59.081934 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:02:59.626295 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:02:59.626264 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:03:02.917765 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:02.917721 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:03.017923 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.017879 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:03.410654 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.410632 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dt99m"] Jul 07 20:03:03.421052 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:03:03.421020 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d270e6_e127_4b3e_a2c0_a7543f1b09fa.slice/crio-8668a5394a16754bbb1245ec86f8efa884311f127d32521b91ef55d7de8ff3b4 WatchSource:0}: Error finding container 8668a5394a16754bbb1245ec86f8efa884311f127d32521b91ef55d7de8ff3b4: Status 404 returned error can't find the container with id 8668a5394a16754bbb1245ec86f8efa884311f127d32521b91ef55d7de8ff3b4 Jul 07 20:03:03.429824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.429755 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59bdc756d5-nldrf"] Jul 07 20:03:03.431905 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:03:03.431878 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef38548_70c7_4cbe_9224_8692f7605752.slice/crio-d92c193614c021313b85fcb2c471eb457fb61617ef4ab96eea64342e7dcd2c6e WatchSource:0}: Error finding container d92c193614c021313b85fcb2c471eb457fb61617ef4ab96eea64342e7dcd2c6e: Status 404 returned error can't find the container with id d92c193614c021313b85fcb2c471eb457fb61617ef4ab96eea64342e7dcd2c6e Jul 07 20:03:03.639884 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.639814 2553 generic.go:356] "Generic (PLEG): container finished" podID="d2bcb0b9-8e9b-4cd1-9633-853e6668363e" containerID="1418b007cef14aa686d57fd6a49cdcec4d44078a34cf0452b4672040d0e2c398" exitCode=0 Jul 07 20:03:03.640068 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.639895 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" event={"ID":"d2bcb0b9-8e9b-4cd1-9633-853e6668363e","Type":"ContainerDied","Data":"1418b007cef14aa686d57fd6a49cdcec4d44078a34cf0452b4672040d0e2c398"} Jul 07 20:03:03.640311 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.640288 2553 scope.go:117] "RemoveContainer" containerID="1418b007cef14aa686d57fd6a49cdcec4d44078a34cf0452b4672040d0e2c398" Jul 07 20:03:03.641264 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.641240 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dt99m" event={"ID":"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa","Type":"ContainerStarted","Data":"8668a5394a16754bbb1245ec86f8efa884311f127d32521b91ef55d7de8ff3b4"} Jul 07 20:03:03.643170 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.643145 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-pt5cw" event={"ID":"6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41","Type":"ContainerStarted","Data":"82c2734f944c4bba74e3088c899d75609652054aea1e2671e9e6caf83925a101"} Jul 07 20:03:03.643324 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.643299 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-pt5cw" Jul 07 20:03:03.645364 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.645344 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bdc756d5-nldrf" event={"ID":"bef38548-70c7-4cbe-9224-8692f7605752","Type":"ContainerStarted","Data":"a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3"} Jul 07 20:03:03.645445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.645380 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bdc756d5-nldrf" event={"ID":"bef38548-70c7-4cbe-9224-8692f7605752","Type":"ContainerStarted","Data":"d92c193614c021313b85fcb2c471eb457fb61617ef4ab96eea64342e7dcd2c6e"} Jul 07 20:03:03.657667 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.657647 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-pt5cw" Jul 07 20:03:03.678635 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.678591 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:03.692815 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.692745 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59bdc756d5-nldrf" podStartSLOduration=6.692721038 podStartE2EDuration="6.692721038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:02:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:03:03.690599158 +0000 UTC m=+136.199271736" watchObservedRunningTime="2026-07-07 20:03:03.692721038 +0000 UTC m=+136.201393615" Jul 07 20:03:03.724974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.724876 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-pt5cw" podStartSLOduration=1.641311613 podStartE2EDuration="25.724852494s" totalImagesPullingTime="24.083540881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:03:03.722379898 +0000 UTC m=+136.231052475" watchObservedRunningTime="2026-07-07 20:03:03.724852494 +0000 UTC m=+136.233525072" Jul 07 20:03:03.765486 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:03.765447 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:04.653489 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:04.653381 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cqz4g" event={"ID":"d2bcb0b9-8e9b-4cd1-9633-853e6668363e","Type":"ContainerStarted","Data":"e92699e110d01efc971fb2d686670456cad4271ac8dcaff6a3d6394b978ae6cd"} Jul 07 20:03:05.661242 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:05.661197 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dt99m" event={"ID":"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa","Type":"ContainerStarted","Data":"c5057e88f8debd0c991a791dd7937a7afe266e7e1bd34d6c9d36f593c0d6d1f8"} Jul 07 20:03:05.661702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:05.661278 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dt99m" event={"ID":"d4d270e6-e127-4b3e-a2c0-a7543f1b09fa","Type":"ContainerStarted","Data":"56a332e3e22392a909ac9ece80ad701d2ee4d6cc2392775b34be9066453cc2c0"} Jul 07 20:03:05.686410 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:05.686336 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dt99m" podStartSLOduration=136.365103596 podStartE2EDuration="2m17.686318155s" totalImagesPullingTime="1.321214559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:03:05.685284135 +0000 UTC m=+138.193956711" watchObservedRunningTime="2026-07-07 20:03:05.686318155 +0000 UTC m=+138.194990787" Jul 07 20:03:05.743457 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:05.743414 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:05.833457 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:05.833413 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:05.954924 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:05.954828 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:06.334081 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:06.334044 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:06.412314 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:06.412273 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:07.850247 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:07.850205 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:03:07.850702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:07.850306 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:03:07.856069 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:07.856043 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:03:08.680424 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:08.680391 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:03:08.776872 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:08.776840 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfb898459-v78qp"] Jul 07 20:03:08.934453 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:08.934367 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:09.040431 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:09.040386 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:09.140283 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:09.140248 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:10.016282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.016240 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" podUID="fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" containerName="registry" containerID="cri-o://bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800" gracePeriod=30 Jul 07 20:03:10.311660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.311631 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:03:10.327187 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327148 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-image-registry-private-configuration\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.327334 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327191 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjxkf\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-kube-api-access-qjxkf\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.327334 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327254 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-trusted-ca\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.327439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327417 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-certificates\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.327495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327479 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-bound-sa-token\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.327551 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327498 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-installation-pull-secrets\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.327551 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327535 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.327654 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327605 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:03:10.328440 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.327800 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-trusted-ca\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.328440 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.328391 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:03:10.330257 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.330209 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:03:10.330379 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.330325 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:03:10.330611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.330583 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-kube-api-access-qjxkf" (OuterVolumeSpecName: "kube-api-access-qjxkf") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "kube-api-access-qjxkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:03:10.330884 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.330853 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:03:10.331207 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.331185 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:03:10.428402 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.428370 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-ca-trust-extracted\") pod \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\" (UID: \"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92\") " Jul 07 20:03:10.428585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.428549 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-certificates\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.428585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.428567 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-bound-sa-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.428585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.428583 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-installation-pull-secrets\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.428738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.428597 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-registry-tls\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.428738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.428613 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-image-registry-private-configuration\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.428738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.428627 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjxkf\" (UniqueName: \"kubernetes.io/projected/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-kube-api-access-qjxkf\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.438689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.438651 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" (UID: "fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:03:10.529928 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.529850 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92-ca-trust-extracted\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:10.686742 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.686699 2553 generic.go:356] "Generic (PLEG): container finished" podID="fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" containerID="bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800" exitCode=0 Jul 07 20:03:10.686945 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.686757 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" event={"ID":"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92","Type":"ContainerDied","Data":"bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800"} Jul 07 20:03:10.686945 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.686760 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" Jul 07 20:03:10.686945 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.686838 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559bc8fb5d-76pcm" event={"ID":"fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92","Type":"ContainerDied","Data":"d583aeafa2030eb7a924daa893c0a593da1eded78ed1585d67900d1c78667b05"} Jul 07 20:03:10.686945 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.686868 2553 scope.go:117] "RemoveContainer" containerID="bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800" Jul 07 20:03:10.697105 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.697079 2553 scope.go:117] "RemoveContainer" containerID="bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800" Jul 07 20:03:10.697441 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:03:10.697414 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800\": container with ID starting with bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800 not found: ID does not exist" containerID="bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800" Jul 07 20:03:10.697527 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.697448 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800"} err="failed to get container status \"bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800\": rpc error: code = NotFound desc = could not find container \"bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800\": container with ID starting with bfc960a73f7df533cfa92044f4eddaefeab631c21df25b863c2f5ef6b692e800 not found: ID does not exist" Jul 07 20:03:10.713772 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.713734 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559bc8fb5d-76pcm"] Jul 07 20:03:10.718558 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:10.718535 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-559bc8fb5d-76pcm"] Jul 07 20:03:12.028508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:12.028471 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" path="/var/lib/kubelet/pods/fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92/volumes" Jul 07 20:03:33.680935 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:33.680895 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:33.744895 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:33.744863 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:33.804064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:33.804018 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6dfb898459-v78qp" podUID="2c46bf51-9e02-48e4-a238-c5e16a0fa425" containerName="console" containerID="cri-o://4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416" gracePeriod=15 Jul 07 20:03:34.079871 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.079839 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:03:34.113846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.113819 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-oauth-config\") pod \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " Jul 07 20:03:34.113985 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.113860 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-oauth-serving-cert\") pod \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " Jul 07 20:03:34.113985 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.113883 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmj9n\" (UniqueName: \"kubernetes.io/projected/2c46bf51-9e02-48e4-a238-c5e16a0fa425-kube-api-access-dmj9n\") pod \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " Jul 07 20:03:34.113985 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.113933 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-config\") pod \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " Jul 07 20:03:34.113985 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.113975 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-serving-cert\") pod \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " Jul 07 20:03:34.114215 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.114009 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-service-ca\") pod \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\" (UID: \"2c46bf51-9e02-48e4-a238-c5e16a0fa425\") " Jul 07 20:03:34.114309 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.114276 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2c46bf51-9e02-48e4-a238-c5e16a0fa425" (UID: "2c46bf51-9e02-48e4-a238-c5e16a0fa425"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:03:34.114396 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.114371 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-config" (OuterVolumeSpecName: "console-config") pod "2c46bf51-9e02-48e4-a238-c5e16a0fa425" (UID: "2c46bf51-9e02-48e4-a238-c5e16a0fa425"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:03:34.114520 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.114494 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-service-ca" (OuterVolumeSpecName: "service-ca") pod "2c46bf51-9e02-48e4-a238-c5e16a0fa425" (UID: "2c46bf51-9e02-48e4-a238-c5e16a0fa425"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:03:34.116203 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.116163 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c46bf51-9e02-48e4-a238-c5e16a0fa425-kube-api-access-dmj9n" (OuterVolumeSpecName: "kube-api-access-dmj9n") pod "2c46bf51-9e02-48e4-a238-c5e16a0fa425" (UID: "2c46bf51-9e02-48e4-a238-c5e16a0fa425"). InnerVolumeSpecName "kube-api-access-dmj9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:03:34.116320 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.116255 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2c46bf51-9e02-48e4-a238-c5e16a0fa425" (UID: "2c46bf51-9e02-48e4-a238-c5e16a0fa425"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:03:34.116399 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.116376 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2c46bf51-9e02-48e4-a238-c5e16a0fa425" (UID: "2c46bf51-9e02-48e4-a238-c5e16a0fa425"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:03:34.214605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.214528 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-oauth-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:34.214605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.214556 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-oauth-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:34.214605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.214570 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmj9n\" (UniqueName: \"kubernetes.io/projected/2c46bf51-9e02-48e4-a238-c5e16a0fa425-kube-api-access-dmj9n\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:34.214605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.214583 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:34.214605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.214595 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c46bf51-9e02-48e4-a238-c5e16a0fa425-console-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:34.214605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.214606 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46bf51-9e02-48e4-a238-c5e16a0fa425-service-ca\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:03:34.778224 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.778194 2553 generic.go:356] "Generic (PLEG): container finished" podID="2c46bf51-9e02-48e4-a238-c5e16a0fa425" containerID="4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416" exitCode=0 Jul 07 20:03:34.778611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.778250 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb898459-v78qp" Jul 07 20:03:34.778611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.778277 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb898459-v78qp" event={"ID":"2c46bf51-9e02-48e4-a238-c5e16a0fa425","Type":"ContainerDied","Data":"4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416"} Jul 07 20:03:34.778611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.778330 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb898459-v78qp" event={"ID":"2c46bf51-9e02-48e4-a238-c5e16a0fa425","Type":"ContainerDied","Data":"158e6758beca5f07e5220b89dd07a26e004e5e78cb4ef7497cfbd3775c6b62ae"} Jul 07 20:03:34.778611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.778353 2553 scope.go:117] "RemoveContainer" containerID="4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416" Jul 07 20:03:34.786948 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.786931 2553 scope.go:117] "RemoveContainer" containerID="4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416" Jul 07 20:03:34.787225 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:03:34.787208 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416\": container with ID starting with 4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416 not found: ID does not exist" containerID="4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416" Jul 07 20:03:34.787270 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.787233 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416"} err="failed to get container status \"4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416\": rpc error: code = NotFound desc = could not find container \"4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416\": container with ID starting with 4b9da6c9465c95bd874815e2e6abb7f38a5bfceee5757a47a9afaa904a76c416 not found: ID does not exist" Jul 07 20:03:34.800754 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.800732 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfb898459-v78qp"] Jul 07 20:03:34.802721 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:34.802701 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dfb898459-v78qp"] Jul 07 20:03:36.028624 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:36.028594 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c46bf51-9e02-48e4-a238-c5e16a0fa425" path="/var/lib/kubelet/pods/2c46bf51-9e02-48e4-a238-c5e16a0fa425/volumes" Jul 07 20:03:44.763590 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:44.763501 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:48.789262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:48.789223 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:48.990712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:48.990673 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:55.130330 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:55.130290 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:58.058762 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.058728 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7449f599b8-7fhcj"] Jul 07 20:03:58.059692 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.059668 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" containerName="registry" Jul 07 20:03:58.059819 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.059805 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" containerName="registry" Jul 07 20:03:58.059922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.059909 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c46bf51-9e02-48e4-a238-c5e16a0fa425" containerName="console" Jul 07 20:03:58.060025 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.060014 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c46bf51-9e02-48e4-a238-c5e16a0fa425" containerName="console" Jul 07 20:03:58.060220 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.060207 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c46bf51-9e02-48e4-a238-c5e16a0fa425" containerName="console" Jul 07 20:03:58.060318 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.060308 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0c9f14-5763-4e1a-b904-9cdbeb8e4b92" containerName="registry" Jul 07 20:03:58.064483 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.064336 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.083097 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.083070 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7449f599b8-7fhcj"] Jul 07 20:03:58.189714 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.189677 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:58.214389 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.214365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-serving-cert\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.214507 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.214393 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-oauth-serving-cert\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.214507 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.214417 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sj7g\" (UniqueName: \"kubernetes.io/projected/ca94e2d9-4039-4eba-9097-e60199159986-kube-api-access-4sj7g\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.214507 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.214475 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-service-ca\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.214507 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.214503 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-oauth-config\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.214653 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.214539 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-trusted-ca-bundle\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.214653 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.214554 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-console-config\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.290609 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.290580 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:03:58.315811 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.315753 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-oauth-config\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.315811 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.315786 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-trusted-ca-bundle\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.315811 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.315805 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-console-config\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.316127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.315842 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-serving-cert\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.316127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.316016 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-oauth-serving-cert\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.316127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.316071 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sj7g\" (UniqueName: \"kubernetes.io/projected/ca94e2d9-4039-4eba-9097-e60199159986-kube-api-access-4sj7g\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.316276 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.316133 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-service-ca\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.316707 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.316675 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-console-config\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.316707 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.316691 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-oauth-serving-cert\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.316950 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.316927 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-service-ca\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.317098 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.317075 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-trusted-ca-bundle\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.318294 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.318268 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-oauth-config\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.318460 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.318443 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-serving-cert\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.327502 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.327479 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sj7g\" (UniqueName: \"kubernetes.io/projected/ca94e2d9-4039-4eba-9097-e60199159986-kube-api-access-4sj7g\") pod \"console-7449f599b8-7fhcj\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.375761 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.375716 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:03:58.521977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.521946 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7449f599b8-7fhcj"] Jul 07 20:03:58.525205 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:03:58.525177 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca94e2d9_4039_4eba_9097_e60199159986.slice/crio-49451970bf952af98116840fcfa05e3e6386cad54481132480fd667e614dd7c3 WatchSource:0}: Error finding container 49451970bf952af98116840fcfa05e3e6386cad54481132480fd667e614dd7c3: Status 404 returned error can't find the container with id 49451970bf952af98116840fcfa05e3e6386cad54481132480fd667e614dd7c3 Jul 07 20:03:58.872643 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.872558 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449f599b8-7fhcj" event={"ID":"ca94e2d9-4039-4eba-9097-e60199159986","Type":"ContainerStarted","Data":"8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7"} Jul 07 20:03:58.872643 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.872608 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449f599b8-7fhcj" event={"ID":"ca94e2d9-4039-4eba-9097-e60199159986","Type":"ContainerStarted","Data":"49451970bf952af98116840fcfa05e3e6386cad54481132480fd667e614dd7c3"} Jul 07 20:03:58.896156 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:03:58.896112 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7449f599b8-7fhcj" podStartSLOduration=0.896096627 podStartE2EDuration="896.096627ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:03:58.894036762 +0000 UTC m=+191.402709338" watchObservedRunningTime="2026-07-07 20:03:58.896096627 +0000 UTC m=+191.404769203" Jul 07 20:04:08.375970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:08.375924 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:04:08.376397 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:08.376012 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:04:08.380818 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:08.380798 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:04:08.915194 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:08.915162 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:04:08.979117 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:08.979079 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59bdc756d5-nldrf"] Jul 07 20:04:09.123435 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:09.123387 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:04:09.222792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:09.222714 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:04:09.300977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:09.300948 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:04:15.706475 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:15.706437 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:04:33.478926 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:33.478890 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:04:34.001000 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.000963 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-59bdc756d5-nldrf" podUID="bef38548-70c7-4cbe-9224-8692f7605752" containerName="console" containerID="cri-o://a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3" gracePeriod=15 Jul 07 20:04:34.240784 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.240760 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:04:34.306143 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306075 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-oauth-serving-cert\") pod \"bef38548-70c7-4cbe-9224-8692f7605752\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " Jul 07 20:04:34.306143 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306128 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-console-config\") pod \"bef38548-70c7-4cbe-9224-8692f7605752\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " Jul 07 20:04:34.306306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306160 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-service-ca\") pod \"bef38548-70c7-4cbe-9224-8692f7605752\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " Jul 07 20:04:34.306306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306197 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-oauth-config\") pod \"bef38548-70c7-4cbe-9224-8692f7605752\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " Jul 07 20:04:34.306306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306224 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84lgv\" (UniqueName: \"kubernetes.io/projected/bef38548-70c7-4cbe-9224-8692f7605752-kube-api-access-84lgv\") pod \"bef38548-70c7-4cbe-9224-8692f7605752\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " Jul 07 20:04:34.306306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306255 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-trusted-ca-bundle\") pod \"bef38548-70c7-4cbe-9224-8692f7605752\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " Jul 07 20:04:34.306306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306291 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-serving-cert\") pod \"bef38548-70c7-4cbe-9224-8692f7605752\" (UID: \"bef38548-70c7-4cbe-9224-8692f7605752\") " Jul 07 20:04:34.306554 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306506 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bef38548-70c7-4cbe-9224-8692f7605752" (UID: "bef38548-70c7-4cbe-9224-8692f7605752"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:04:34.306611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306569 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-console-config" (OuterVolumeSpecName: "console-config") pod "bef38548-70c7-4cbe-9224-8692f7605752" (UID: "bef38548-70c7-4cbe-9224-8692f7605752"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:04:34.306669 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306620 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-service-ca" (OuterVolumeSpecName: "service-ca") pod "bef38548-70c7-4cbe-9224-8692f7605752" (UID: "bef38548-70c7-4cbe-9224-8692f7605752"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:04:34.306727 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306670 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-oauth-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:04:34.306727 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306682 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-console-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:04:34.306827 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.306766 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bef38548-70c7-4cbe-9224-8692f7605752" (UID: "bef38548-70c7-4cbe-9224-8692f7605752"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:04:34.308391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.308365 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bef38548-70c7-4cbe-9224-8692f7605752" (UID: "bef38548-70c7-4cbe-9224-8692f7605752"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:04:34.308474 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.308385 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bef38548-70c7-4cbe-9224-8692f7605752" (UID: "bef38548-70c7-4cbe-9224-8692f7605752"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:04:34.308474 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.308461 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bef38548-70c7-4cbe-9224-8692f7605752-kube-api-access-84lgv" (OuterVolumeSpecName: "kube-api-access-84lgv") pod "bef38548-70c7-4cbe-9224-8692f7605752" (UID: "bef38548-70c7-4cbe-9224-8692f7605752"). InnerVolumeSpecName "kube-api-access-84lgv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:04:34.407775 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.407749 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:04:34.407775 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.407772 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-service-ca\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:04:34.407906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.407781 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bef38548-70c7-4cbe-9224-8692f7605752-console-oauth-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:04:34.407906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.407791 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84lgv\" (UniqueName: \"kubernetes.io/projected/bef38548-70c7-4cbe-9224-8692f7605752-kube-api-access-84lgv\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:04:34.407906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:34.407799 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bef38548-70c7-4cbe-9224-8692f7605752-trusted-ca-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:04:35.005742 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.005708 2553 generic.go:356] "Generic (PLEG): container finished" podID="bef38548-70c7-4cbe-9224-8692f7605752" containerID="a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3" exitCode=0 Jul 07 20:04:35.006172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.005750 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59bdc756d5-nldrf" Jul 07 20:04:35.006172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.005798 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bdc756d5-nldrf" event={"ID":"bef38548-70c7-4cbe-9224-8692f7605752","Type":"ContainerDied","Data":"a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3"} Jul 07 20:04:35.006172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.005844 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59bdc756d5-nldrf" event={"ID":"bef38548-70c7-4cbe-9224-8692f7605752","Type":"ContainerDied","Data":"d92c193614c021313b85fcb2c471eb457fb61617ef4ab96eea64342e7dcd2c6e"} Jul 07 20:04:35.006172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.005865 2553 scope.go:117] "RemoveContainer" containerID="a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3" Jul 07 20:04:35.014103 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.014069 2553 scope.go:117] "RemoveContainer" containerID="a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3" Jul 07 20:04:35.014346 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:04:35.014328 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3\": container with ID starting with a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3 not found: ID does not exist" containerID="a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3" Jul 07 20:04:35.014412 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.014355 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3"} err="failed to get container status \"a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3\": rpc error: code = NotFound desc = could not find container \"a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3\": container with ID starting with a1540ccc13ea867f405f3821e0ef06c26027dcebf892a3a631b6bc7fa09444b3 not found: ID does not exist" Jul 07 20:04:35.032644 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.032618 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59bdc756d5-nldrf"] Jul 07 20:04:35.035782 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:35.035762 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59bdc756d5-nldrf"] Jul 07 20:04:36.027596 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:04:36.027562 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bef38548-70c7-4cbe-9224-8692f7605752" path="/var/lib/kubelet/pods/bef38548-70c7-4cbe-9224-8692f7605752/volumes" Jul 07 20:05:04.058681 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:04.058637 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:04.120062 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:04.120010 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:16.070184 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:16.070098 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:16.136114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:16.136075 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:18.752748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:18.752710 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:18.864378 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:18.864344 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:34.961538 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:34.961508 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77c84db5bb-59prz"] Jul 07 20:05:34.961972 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:34.961788 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bef38548-70c7-4cbe-9224-8692f7605752" containerName="console" Jul 07 20:05:34.961972 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:34.961798 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef38548-70c7-4cbe-9224-8692f7605752" containerName="console" Jul 07 20:05:34.961972 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:34.961847 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="bef38548-70c7-4cbe-9224-8692f7605752" containerName="console" Jul 07 20:05:34.964917 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:34.964891 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:34.976530 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:34.976492 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c84db5bb-59prz"] Jul 07 20:05:35.019782 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.019746 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:35.065647 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.065611 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-serving-cert\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.065804 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.065680 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-trusted-ca-bundle\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.065804 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.065713 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-oauth-config\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.065804 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.065767 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-config\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.065804 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.065788 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-service-ca\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.066008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.065814 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgsdt\" (UniqueName: \"kubernetes.io/projected/ed0ec38e-ff71-43c5-8690-1c629bc895eb-kube-api-access-xgsdt\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.066008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.065949 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-oauth-serving-cert\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.131309 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.131275 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:35.166495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.166464 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-oauth-serving-cert\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.166637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.166517 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-serving-cert\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.166637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.166562 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-trusted-ca-bundle\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.166637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.166595 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-oauth-config\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.166637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.166626 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-config\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.166834 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.166648 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-service-ca\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.166834 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.166674 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgsdt\" (UniqueName: \"kubernetes.io/projected/ed0ec38e-ff71-43c5-8690-1c629bc895eb-kube-api-access-xgsdt\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.167347 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.167315 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-oauth-serving-cert\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.167465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.167328 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-config\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.167465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.167395 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-service-ca\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.167580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.167550 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-trusted-ca-bundle\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.169051 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.169029 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-serving-cert\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.169139 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.169094 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-oauth-config\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.182661 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.182642 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgsdt\" (UniqueName: \"kubernetes.io/projected/ed0ec38e-ff71-43c5-8690-1c629bc895eb-kube-api-access-xgsdt\") pod \"console-77c84db5bb-59prz\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.222398 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.222338 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:35.274215 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.274195 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:35.405086 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.405045 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c84db5bb-59prz"] Jul 07 20:05:35.408215 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:05:35.408189 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0ec38e_ff71_43c5_8690_1c629bc895eb.slice/crio-f1bba816a44ed5e33795b2bd03e791dabee95f2ee0feafce5e4569dd6b981537 WatchSource:0}: Error finding container f1bba816a44ed5e33795b2bd03e791dabee95f2ee0feafce5e4569dd6b981537: Status 404 returned error can't find the container with id f1bba816a44ed5e33795b2bd03e791dabee95f2ee0feafce5e4569dd6b981537 Jul 07 20:05:35.818058 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.818021 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:35.876108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:35.876077 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:36.230478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:36.230446 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c84db5bb-59prz" event={"ID":"ed0ec38e-ff71-43c5-8690-1c629bc895eb","Type":"ContainerStarted","Data":"12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b"} Jul 07 20:05:36.230910 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:36.230492 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c84db5bb-59prz" event={"ID":"ed0ec38e-ff71-43c5-8690-1c629bc895eb","Type":"ContainerStarted","Data":"f1bba816a44ed5e33795b2bd03e791dabee95f2ee0feafce5e4569dd6b981537"} Jul 07 20:05:36.253547 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:36.253496 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77c84db5bb-59prz" podStartSLOduration=2.25347953 podStartE2EDuration="2.25347953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:05:36.252216044 +0000 UTC m=+288.760888613" watchObservedRunningTime="2026-07-07 20:05:36.25347953 +0000 UTC m=+288.762152106" Jul 07 20:05:45.274533 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:45.274502 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:45.274894 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:45.274552 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:45.279101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:45.279078 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:46.271674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.271646 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:05:46.325413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.325385 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7449f599b8-7fhcj"] Jul 07 20:05:46.431862 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.431826 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:46.531934 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.531845 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:46.632628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.632585 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:05:46.993821 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.993788 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2"] Jul 07 20:05:46.997136 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.997118 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:46.999961 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.999938 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 20:05:47.000092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:46.999964 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 20:05:47.000092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.000027 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-256hp\"" Jul 07 20:05:47.009411 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.009391 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2"] Jul 07 20:05:47.067585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.067548 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.067585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.067592 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.067784 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.067665 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m24h\" (UniqueName: \"kubernetes.io/projected/b3319065-9a8e-4f29-9c90-02aa40f1e735-kube-api-access-6m24h\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.168395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.168364 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.168395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.168396 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.168548 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.168424 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m24h\" (UniqueName: \"kubernetes.io/projected/b3319065-9a8e-4f29-9c90-02aa40f1e735-kube-api-access-6m24h\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.168828 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.168807 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.168868 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.168816 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.178226 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.178204 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m24h\" (UniqueName: \"kubernetes.io/projected/b3319065-9a8e-4f29-9c90-02aa40f1e735-kube-api-access-6m24h\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.306954 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.306872 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:05:47.434828 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.434802 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2"] Jul 07 20:05:47.437440 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:05:47.437403 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3319065_9a8e_4f29_9c90_02aa40f1e735.slice/crio-418492dbf70ba67b96e98dedc9b98939dac69a6d5732aa7e83ac14d60937a1e6 WatchSource:0}: Error finding container 418492dbf70ba67b96e98dedc9b98939dac69a6d5732aa7e83ac14d60937a1e6: Status 404 returned error can't find the container with id 418492dbf70ba67b96e98dedc9b98939dac69a6d5732aa7e83ac14d60937a1e6 Jul 07 20:05:47.916158 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.916132 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:05:47.924651 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.924618 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:05:47.924929 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:47.924909 2553 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 20:05:48.276617 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:48.276580 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" event={"ID":"b3319065-9a8e-4f29-9c90-02aa40f1e735","Type":"ContainerStarted","Data":"418492dbf70ba67b96e98dedc9b98939dac69a6d5732aa7e83ac14d60937a1e6"} Jul 07 20:05:56.308883 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:56.308845 2553 generic.go:356] "Generic (PLEG): container finished" podID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerID="35d5daf420397e5b80d8db24f1a1c1e1774d6c7b72637dda3882947dc5750405" exitCode=0 Jul 07 20:05:56.309280 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:56.308891 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" event={"ID":"b3319065-9a8e-4f29-9c90-02aa40f1e735","Type":"ContainerDied","Data":"35d5daf420397e5b80d8db24f1a1c1e1774d6c7b72637dda3882947dc5750405"} Jul 07 20:05:56.309917 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:05:56.309896 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 20:06:00.326455 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:00.326417 2553 generic.go:356] "Generic (PLEG): container finished" podID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerID="d81adab2db4adc0e05b49295a6bfee7162164f3fcb24cdb259c12adb8d8235f2" exitCode=0 Jul 07 20:06:00.326455 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:00.326455 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" event={"ID":"b3319065-9a8e-4f29-9c90-02aa40f1e735","Type":"ContainerDied","Data":"d81adab2db4adc0e05b49295a6bfee7162164f3fcb24cdb259c12adb8d8235f2"} Jul 07 20:06:09.375345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:09.375310 2553 generic.go:356] "Generic (PLEG): container finished" podID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerID="d7abf835fed6da4b87f6f0b056f4810888e2e4b2b397fd65e3fff359835329fd" exitCode=0 Jul 07 20:06:09.375738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:09.375352 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" event={"ID":"b3319065-9a8e-4f29-9c90-02aa40f1e735","Type":"ContainerDied","Data":"d7abf835fed6da4b87f6f0b056f4810888e2e4b2b397fd65e3fff359835329fd"} Jul 07 20:06:10.500693 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.500668 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:06:10.673488 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.673410 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-bundle\") pod \"b3319065-9a8e-4f29-9c90-02aa40f1e735\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " Jul 07 20:06:10.673488 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.673449 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m24h\" (UniqueName: \"kubernetes.io/projected/b3319065-9a8e-4f29-9c90-02aa40f1e735-kube-api-access-6m24h\") pod \"b3319065-9a8e-4f29-9c90-02aa40f1e735\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " Jul 07 20:06:10.673488 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.673477 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-util\") pod \"b3319065-9a8e-4f29-9c90-02aa40f1e735\" (UID: \"b3319065-9a8e-4f29-9c90-02aa40f1e735\") " Jul 07 20:06:10.674285 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.674257 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-bundle" (OuterVolumeSpecName: "bundle") pod "b3319065-9a8e-4f29-9c90-02aa40f1e735" (UID: "b3319065-9a8e-4f29-9c90-02aa40f1e735"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:06:10.675588 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.675553 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3319065-9a8e-4f29-9c90-02aa40f1e735-kube-api-access-6m24h" (OuterVolumeSpecName: "kube-api-access-6m24h") pod "b3319065-9a8e-4f29-9c90-02aa40f1e735" (UID: "b3319065-9a8e-4f29-9c90-02aa40f1e735"). InnerVolumeSpecName "kube-api-access-6m24h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:06:10.677899 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.677879 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-util" (OuterVolumeSpecName: "util") pod "b3319065-9a8e-4f29-9c90-02aa40f1e735" (UID: "b3319065-9a8e-4f29-9c90-02aa40f1e735"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:06:10.774255 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.774222 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:10.774255 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.774249 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m24h\" (UniqueName: \"kubernetes.io/projected/b3319065-9a8e-4f29-9c90-02aa40f1e735-kube-api-access-6m24h\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:10.774255 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:10.774260 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3319065-9a8e-4f29-9c90-02aa40f1e735-util\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:11.348046 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.347956 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7449f599b8-7fhcj" podUID="ca94e2d9-4039-4eba-9097-e60199159986" containerName="console" containerID="cri-o://8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7" gracePeriod=15 Jul 07 20:06:11.384511 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.384492 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" Jul 07 20:06:11.384636 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.384487 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540r2kx2" event={"ID":"b3319065-9a8e-4f29-9c90-02aa40f1e735","Type":"ContainerDied","Data":"418492dbf70ba67b96e98dedc9b98939dac69a6d5732aa7e83ac14d60937a1e6"} Jul 07 20:06:11.384636 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.384608 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="418492dbf70ba67b96e98dedc9b98939dac69a6d5732aa7e83ac14d60937a1e6" Jul 07 20:06:11.590615 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.590589 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:06:11.680903 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.680830 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-oauth-config\") pod \"ca94e2d9-4039-4eba-9097-e60199159986\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " Jul 07 20:06:11.680903 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.680866 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-service-ca\") pod \"ca94e2d9-4039-4eba-9097-e60199159986\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " Jul 07 20:06:11.680903 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.680905 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-serving-cert\") pod \"ca94e2d9-4039-4eba-9097-e60199159986\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " Jul 07 20:06:11.681175 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.680926 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sj7g\" (UniqueName: \"kubernetes.io/projected/ca94e2d9-4039-4eba-9097-e60199159986-kube-api-access-4sj7g\") pod \"ca94e2d9-4039-4eba-9097-e60199159986\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " Jul 07 20:06:11.681175 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.680944 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-trusted-ca-bundle\") pod \"ca94e2d9-4039-4eba-9097-e60199159986\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " Jul 07 20:06:11.681175 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.680972 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-oauth-serving-cert\") pod \"ca94e2d9-4039-4eba-9097-e60199159986\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " Jul 07 20:06:11.681175 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.681034 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-console-config\") pod \"ca94e2d9-4039-4eba-9097-e60199159986\" (UID: \"ca94e2d9-4039-4eba-9097-e60199159986\") " Jul 07 20:06:11.681464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.681440 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ca94e2d9-4039-4eba-9097-e60199159986" (UID: "ca94e2d9-4039-4eba-9097-e60199159986"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:06:11.681535 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.681457 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ca94e2d9-4039-4eba-9097-e60199159986" (UID: "ca94e2d9-4039-4eba-9097-e60199159986"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:06:11.681535 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.681490 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-service-ca" (OuterVolumeSpecName: "service-ca") pod "ca94e2d9-4039-4eba-9097-e60199159986" (UID: "ca94e2d9-4039-4eba-9097-e60199159986"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:06:11.681535 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.681525 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-console-config" (OuterVolumeSpecName: "console-config") pod "ca94e2d9-4039-4eba-9097-e60199159986" (UID: "ca94e2d9-4039-4eba-9097-e60199159986"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:06:11.683020 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.682986 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ca94e2d9-4039-4eba-9097-e60199159986" (UID: "ca94e2d9-4039-4eba-9097-e60199159986"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:06:11.683436 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.683420 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca94e2d9-4039-4eba-9097-e60199159986-kube-api-access-4sj7g" (OuterVolumeSpecName: "kube-api-access-4sj7g") pod "ca94e2d9-4039-4eba-9097-e60199159986" (UID: "ca94e2d9-4039-4eba-9097-e60199159986"). InnerVolumeSpecName "kube-api-access-4sj7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:06:11.683436 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.683425 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ca94e2d9-4039-4eba-9097-e60199159986" (UID: "ca94e2d9-4039-4eba-9097-e60199159986"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:06:11.782136 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.782106 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:11.782136 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.782131 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sj7g\" (UniqueName: \"kubernetes.io/projected/ca94e2d9-4039-4eba-9097-e60199159986-kube-api-access-4sj7g\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:11.782136 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.782140 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-trusted-ca-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:11.782321 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.782149 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-oauth-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:11.782321 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.782158 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-console-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:11.782321 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.782166 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca94e2d9-4039-4eba-9097-e60199159986-console-oauth-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:11.782321 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:11.782174 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca94e2d9-4039-4eba-9097-e60199159986-service-ca\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:12.389902 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.389823 2553 generic.go:356] "Generic (PLEG): container finished" podID="ca94e2d9-4039-4eba-9097-e60199159986" containerID="8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7" exitCode=0 Jul 07 20:06:12.389902 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.389863 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449f599b8-7fhcj" Jul 07 20:06:12.389902 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.389887 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449f599b8-7fhcj" event={"ID":"ca94e2d9-4039-4eba-9097-e60199159986","Type":"ContainerDied","Data":"8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7"} Jul 07 20:06:12.390130 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.389930 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449f599b8-7fhcj" event={"ID":"ca94e2d9-4039-4eba-9097-e60199159986","Type":"ContainerDied","Data":"49451970bf952af98116840fcfa05e3e6386cad54481132480fd667e614dd7c3"} Jul 07 20:06:12.390130 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.389952 2553 scope.go:117] "RemoveContainer" containerID="8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7" Jul 07 20:06:12.397921 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.397897 2553 scope.go:117] "RemoveContainer" containerID="8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7" Jul 07 20:06:12.398160 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:06:12.398144 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7\": container with ID starting with 8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7 not found: ID does not exist" containerID="8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7" Jul 07 20:06:12.398208 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.398167 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7"} err="failed to get container status \"8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7\": rpc error: code = NotFound desc = could not find container \"8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7\": container with ID starting with 8ca8b81b8ecd3ddc8af2a1a24fde2cccf24a9ac849ea029fec9c1406a92020c7 not found: ID does not exist" Jul 07 20:06:12.410001 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.409975 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7449f599b8-7fhcj"] Jul 07 20:06:12.414883 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:12.414862 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7449f599b8-7fhcj"] Jul 07 20:06:13.018210 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:13.018177 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:14.027813 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.027781 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca94e2d9-4039-4eba-9097-e60199159986" path="/var/lib/kubelet/pods/ca94e2d9-4039-4eba-9097-e60199159986/volumes" Jul 07 20:06:14.032138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.032109 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:14.918959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.918922 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh"] Jul 07 20:06:14.919245 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919233 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerName="extract" Jul 07 20:06:14.919300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919247 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerName="extract" Jul 07 20:06:14.919300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919259 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca94e2d9-4039-4eba-9097-e60199159986" containerName="console" Jul 07 20:06:14.919300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919265 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca94e2d9-4039-4eba-9097-e60199159986" containerName="console" Jul 07 20:06:14.919300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919270 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerName="pull" Jul 07 20:06:14.919300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919276 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerName="pull" Jul 07 20:06:14.919300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919288 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerName="util" Jul 07 20:06:14.919300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919297 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerName="util" Jul 07 20:06:14.919506 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919344 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca94e2d9-4039-4eba-9097-e60199159986" containerName="console" Jul 07 20:06:14.919506 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.919353 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3319065-9a8e-4f29-9c90-02aa40f1e735" containerName="extract" Jul 07 20:06:14.974010 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.973955 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh"] Jul 07 20:06:14.974162 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.974089 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:14.977174 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.977134 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-kd62s\"" Jul 07 20:06:14.977174 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.977137 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:06:14.977433 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:14.977416 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 20:06:15.104795 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.104758 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxk8f\" (UniqueName: \"kubernetes.io/projected/d41d677d-4af8-49ca-874f-c1bc642c82c0-kube-api-access-jxk8f\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ndgh\" (UID: \"d41d677d-4af8-49ca-874f-c1bc642c82c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:15.105191 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.104870 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d41d677d-4af8-49ca-874f-c1bc642c82c0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ndgh\" (UID: \"d41d677d-4af8-49ca-874f-c1bc642c82c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:15.205957 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.205878 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d41d677d-4af8-49ca-874f-c1bc642c82c0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ndgh\" (UID: \"d41d677d-4af8-49ca-874f-c1bc642c82c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:15.205957 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.205936 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxk8f\" (UniqueName: \"kubernetes.io/projected/d41d677d-4af8-49ca-874f-c1bc642c82c0-kube-api-access-jxk8f\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ndgh\" (UID: \"d41d677d-4af8-49ca-874f-c1bc642c82c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:15.206292 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.206261 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d41d677d-4af8-49ca-874f-c1bc642c82c0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ndgh\" (UID: \"d41d677d-4af8-49ca-874f-c1bc642c82c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:15.217497 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.217469 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxk8f\" (UniqueName: \"kubernetes.io/projected/d41d677d-4af8-49ca-874f-c1bc642c82c0-kube-api-access-jxk8f\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ndgh\" (UID: \"d41d677d-4af8-49ca-874f-c1bc642c82c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:15.286057 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.286033 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" Jul 07 20:06:15.431793 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:15.431772 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh"] Jul 07 20:06:15.433276 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:06:15.433247 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41d677d_4af8_49ca_874f_c1bc642c82c0.slice/crio-fbbacdafef3d4e897119dcb9d8e2dfd071ddf00f24c7876966ddffa3b04b6a5d WatchSource:0}: Error finding container fbbacdafef3d4e897119dcb9d8e2dfd071ddf00f24c7876966ddffa3b04b6a5d: Status 404 returned error can't find the container with id fbbacdafef3d4e897119dcb9d8e2dfd071ddf00f24c7876966ddffa3b04b6a5d Jul 07 20:06:16.409829 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:16.409495 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" event={"ID":"d41d677d-4af8-49ca-874f-c1bc642c82c0","Type":"ContainerStarted","Data":"fbbacdafef3d4e897119dcb9d8e2dfd071ddf00f24c7876966ddffa3b04b6a5d"} Jul 07 20:06:16.658155 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:16.658085 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:16.763305 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:16.763271 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:17.416276 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:17.416244 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" event={"ID":"d41d677d-4af8-49ca-874f-c1bc642c82c0","Type":"ContainerStarted","Data":"aa88c1b20c934497f6a2caa42682b061f4bdc4921270271bc7d4fe3001876c2b"} Jul 07 20:06:17.925796 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:17.925749 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:18.044509 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.044469 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:18.785163 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.785132 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:18.881354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.881315 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:18.882231 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.882179 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ndgh" podStartSLOduration=3.358301303 podStartE2EDuration="4.882158076s" totalImagesPullingTime="1.523856773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:06:17.462726703 +0000 UTC m=+329.971399277" watchObservedRunningTime="2026-07-07 20:06:18.882158076 +0000 UTC m=+331.390830657" Jul 07 20:06:18.882648 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.882630 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7"] Jul 07 20:06:18.886693 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.886674 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:18.889244 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.889215 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-256hp\"" Jul 07 20:06:18.889497 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.889468 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 20:06:18.890419 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.890403 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 20:06:18.894965 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:18.894943 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7"] Jul 07 20:06:19.039194 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.039097 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fqdv\" (UniqueName: \"kubernetes.io/projected/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-kube-api-access-2fqdv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.039194 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.039168 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.039473 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.039215 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.140392 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.140360 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fqdv\" (UniqueName: \"kubernetes.io/projected/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-kube-api-access-2fqdv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.140551 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.140405 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.140551 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.140436 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.140763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.140745 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.140798 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.140774 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.150281 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.150258 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fqdv\" (UniqueName: \"kubernetes.io/projected/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-kube-api-access-2fqdv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.197692 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.197645 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:19.334441 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.334416 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7"] Jul 07 20:06:19.336861 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:06:19.336834 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc71840_ea52_4d0f_9f90_2c6eb83ba58d.slice/crio-74851737e854313f437fb60b794a2e3bf298c52529a535c2d184cc4fe8ee9ce8 WatchSource:0}: Error finding container 74851737e854313f437fb60b794a2e3bf298c52529a535c2d184cc4fe8ee9ce8: Status 404 returned error can't find the container with id 74851737e854313f437fb60b794a2e3bf298c52529a535c2d184cc4fe8ee9ce8 Jul 07 20:06:19.432050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.432013 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" event={"ID":"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d","Type":"ContainerStarted","Data":"44a147c3a439fa54ebe55944dbc1794df18d40e79e9d0de50a278bb163661c69"} Jul 07 20:06:19.432238 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.432090 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" event={"ID":"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d","Type":"ContainerStarted","Data":"74851737e854313f437fb60b794a2e3bf298c52529a535c2d184cc4fe8ee9ce8"} Jul 07 20:06:19.881036 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.880984 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5p8v6"] Jul 07 20:06:19.884251 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.884236 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:19.887016 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.886967 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 20:06:19.887016 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.886974 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rlzdc\"" Jul 07 20:06:19.888343 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.888327 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 20:06:19.893084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:19.893065 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5p8v6"] Jul 07 20:06:20.048816 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.048785 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9348e13-d808-4dc6-829b-9633619ca941-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5p8v6\" (UID: \"e9348e13-d808-4dc6-829b-9633619ca941\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:20.048975 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.048912 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68lsc\" (UniqueName: \"kubernetes.io/projected/e9348e13-d808-4dc6-829b-9633619ca941-kube-api-access-68lsc\") pod \"cert-manager-webhook-6486ffd997-5p8v6\" (UID: \"e9348e13-d808-4dc6-829b-9633619ca941\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:20.149913 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.149837 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68lsc\" (UniqueName: \"kubernetes.io/projected/e9348e13-d808-4dc6-829b-9633619ca941-kube-api-access-68lsc\") pod \"cert-manager-webhook-6486ffd997-5p8v6\" (UID: \"e9348e13-d808-4dc6-829b-9633619ca941\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:20.149913 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.149876 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9348e13-d808-4dc6-829b-9633619ca941-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5p8v6\" (UID: \"e9348e13-d808-4dc6-829b-9633619ca941\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:20.159201 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.159176 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9348e13-d808-4dc6-829b-9633619ca941-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5p8v6\" (UID: \"e9348e13-d808-4dc6-829b-9633619ca941\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:20.159302 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.159283 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68lsc\" (UniqueName: \"kubernetes.io/projected/e9348e13-d808-4dc6-829b-9633619ca941-kube-api-access-68lsc\") pod \"cert-manager-webhook-6486ffd997-5p8v6\" (UID: \"e9348e13-d808-4dc6-829b-9633619ca941\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:20.204001 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.203972 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:20.331250 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.331203 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5p8v6"] Jul 07 20:06:20.333316 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:06:20.333284 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9348e13_d808_4dc6_829b_9633619ca941.slice/crio-ba2fd7e43f5c21afd11878e42b190903b9104b04da076e5e39778abe7c959822 WatchSource:0}: Error finding container ba2fd7e43f5c21afd11878e42b190903b9104b04da076e5e39778abe7c959822: Status 404 returned error can't find the container with id ba2fd7e43f5c21afd11878e42b190903b9104b04da076e5e39778abe7c959822 Jul 07 20:06:20.437540 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.437464 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" event={"ID":"e9348e13-d808-4dc6-829b-9633619ca941","Type":"ContainerStarted","Data":"ba2fd7e43f5c21afd11878e42b190903b9104b04da076e5e39778abe7c959822"} Jul 07 20:06:20.438738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.438711 2553 generic.go:356] "Generic (PLEG): container finished" podID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerID="44a147c3a439fa54ebe55944dbc1794df18d40e79e9d0de50a278bb163661c69" exitCode=0 Jul 07 20:06:20.438835 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.438776 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" event={"ID":"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d","Type":"ContainerDied","Data":"44a147c3a439fa54ebe55944dbc1794df18d40e79e9d0de50a278bb163661c69"} Jul 07 20:06:20.824924 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:20.824893 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:21.186662 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:21.186583 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:23.669852 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:23.669809 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:23.761627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:23.761576 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:24.459654 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:24.459623 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" event={"ID":"e9348e13-d808-4dc6-829b-9633619ca941","Type":"ContainerStarted","Data":"2376f70875e2b6a0a8873bfce0de9cd7910de5a4ec3a2006a89f9b5c80112171"} Jul 07 20:06:24.459856 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:24.459687 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:24.461469 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:24.461444 2553 generic.go:356] "Generic (PLEG): container finished" podID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerID="ccf9fefd340a9d1e85ec4c81972b2f456c7e6f4374858c76dd8ccebf677c060b" exitCode=0 Jul 07 20:06:24.461561 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:24.461478 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" event={"ID":"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d","Type":"ContainerDied","Data":"ccf9fefd340a9d1e85ec4c81972b2f456c7e6f4374858c76dd8ccebf677c060b"} Jul 07 20:06:24.488463 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:24.488422 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" podStartSLOduration=1.8818614089999999 podStartE2EDuration="5.488407917s" totalImagesPullingTime="3.606546508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:06:24.487096634 +0000 UTC m=+336.995769210" watchObservedRunningTime="2026-07-07 20:06:24.488407917 +0000 UTC m=+336.997080492" Jul 07 20:06:24.594439 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:06:24.594414 2553 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc71840_ea52_4d0f_9f90_2c6eb83ba58d.slice/crio-c0a5b59eecd52411cbb568c970507454828c02733b5c7f8b5df6ed602fe80dab.scope\": RecentStats: unable to find data in memory cache]" Jul 07 20:06:25.467215 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:25.467176 2553 generic.go:356] "Generic (PLEG): container finished" podID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerID="c0a5b59eecd52411cbb568c970507454828c02733b5c7f8b5df6ed602fe80dab" exitCode=0 Jul 07 20:06:25.467595 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:25.467247 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" event={"ID":"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d","Type":"ContainerDied","Data":"c0a5b59eecd52411cbb568c970507454828c02733b5c7f8b5df6ed602fe80dab"} Jul 07 20:06:26.599394 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.599371 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:26.703354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.703281 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fqdv\" (UniqueName: \"kubernetes.io/projected/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-kube-api-access-2fqdv\") pod \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " Jul 07 20:06:26.703354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.703317 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-bundle\") pod \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " Jul 07 20:06:26.703521 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.703371 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-util\") pod \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\" (UID: \"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d\") " Jul 07 20:06:26.703738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.703717 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-bundle" (OuterVolumeSpecName: "bundle") pod "6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" (UID: "6fc71840-ea52-4d0f-9f90-2c6eb83ba58d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:06:26.705437 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.705408 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-kube-api-access-2fqdv" (OuterVolumeSpecName: "kube-api-access-2fqdv") pod "6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" (UID: "6fc71840-ea52-4d0f-9f90-2c6eb83ba58d"). InnerVolumeSpecName "kube-api-access-2fqdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:06:26.708062 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.708040 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-util" (OuterVolumeSpecName: "util") pod "6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" (UID: "6fc71840-ea52-4d0f-9f90-2c6eb83ba58d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:06:26.803958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.803915 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fqdv\" (UniqueName: \"kubernetes.io/projected/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-kube-api-access-2fqdv\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:26.803958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.803953 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:26.803958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:26.803962 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fc71840-ea52-4d0f-9f90-2c6eb83ba58d-util\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:27.478222 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:27.478192 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" Jul 07 20:06:27.478395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:27.478191 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fttcx7" event={"ID":"6fc71840-ea52-4d0f-9f90-2c6eb83ba58d","Type":"ContainerDied","Data":"74851737e854313f437fb60b794a2e3bf298c52529a535c2d184cc4fe8ee9ce8"} Jul 07 20:06:27.478395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:27.478306 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74851737e854313f437fb60b794a2e3bf298c52529a535c2d184cc4fe8ee9ce8" Jul 07 20:06:29.407849 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:29.407817 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:30.469783 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:30.469752 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5p8v6" Jul 07 20:06:31.252143 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:31.252112 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:31.324113 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:31.324057 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:35.030509 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:35.030453 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:35.131520 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:35.131483 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:39.504866 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.504830 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-gbmbg"] Jul 07 20:06:39.505263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.505120 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerName="util" Jul 07 20:06:39.505263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.505131 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerName="util" Jul 07 20:06:39.505263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.505151 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerName="extract" Jul 07 20:06:39.505263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.505157 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerName="extract" Jul 07 20:06:39.505263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.505166 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerName="pull" Jul 07 20:06:39.505263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.505171 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerName="pull" Jul 07 20:06:39.505263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.505217 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fc71840-ea52-4d0f-9f90-2c6eb83ba58d" containerName="extract" Jul 07 20:06:39.508835 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.508818 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.512899 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.512878 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jgnf6\"" Jul 07 20:06:39.519768 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.519747 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-gbmbg"] Jul 07 20:06:39.602881 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.602849 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caccb4f4-42da-41fa-a1af-08b3f7794b16-bound-sa-token\") pod \"cert-manager-55b4d954d5-gbmbg\" (UID: \"caccb4f4-42da-41fa-a1af-08b3f7794b16\") " pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.603064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.602919 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4chs\" (UniqueName: \"kubernetes.io/projected/caccb4f4-42da-41fa-a1af-08b3f7794b16-kube-api-access-t4chs\") pod \"cert-manager-55b4d954d5-gbmbg\" (UID: \"caccb4f4-42da-41fa-a1af-08b3f7794b16\") " pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.704286 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.704247 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4chs\" (UniqueName: \"kubernetes.io/projected/caccb4f4-42da-41fa-a1af-08b3f7794b16-kube-api-access-t4chs\") pod \"cert-manager-55b4d954d5-gbmbg\" (UID: \"caccb4f4-42da-41fa-a1af-08b3f7794b16\") " pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.704423 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.704298 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caccb4f4-42da-41fa-a1af-08b3f7794b16-bound-sa-token\") pod \"cert-manager-55b4d954d5-gbmbg\" (UID: \"caccb4f4-42da-41fa-a1af-08b3f7794b16\") " pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.713480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.713452 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caccb4f4-42da-41fa-a1af-08b3f7794b16-bound-sa-token\") pod \"cert-manager-55b4d954d5-gbmbg\" (UID: \"caccb4f4-42da-41fa-a1af-08b3f7794b16\") " pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.713599 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.713571 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4chs\" (UniqueName: \"kubernetes.io/projected/caccb4f4-42da-41fa-a1af-08b3f7794b16-kube-api-access-t4chs\") pod \"cert-manager-55b4d954d5-gbmbg\" (UID: \"caccb4f4-42da-41fa-a1af-08b3f7794b16\") " pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.817881 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.817797 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-gbmbg" Jul 07 20:06:39.946931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:39.946910 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-gbmbg"] Jul 07 20:06:39.948805 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:06:39.948774 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcaccb4f4_42da_41fa_a1af_08b3f7794b16.slice/crio-ed79985fb32104d5e6a71fba1884099376900fbac9110a6fd3146c3c8a2cc414 WatchSource:0}: Error finding container ed79985fb32104d5e6a71fba1884099376900fbac9110a6fd3146c3c8a2cc414: Status 404 returned error can't find the container with id ed79985fb32104d5e6a71fba1884099376900fbac9110a6fd3146c3c8a2cc414 Jul 07 20:06:40.534939 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:40.534856 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-gbmbg" event={"ID":"caccb4f4-42da-41fa-a1af-08b3f7794b16","Type":"ContainerStarted","Data":"c530e3368652a7498d70fdb51d2090ec3c74301324644f860351c6aee4438494"} Jul 07 20:06:40.534939 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:40.534900 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-gbmbg" event={"ID":"caccb4f4-42da-41fa-a1af-08b3f7794b16","Type":"ContainerStarted","Data":"ed79985fb32104d5e6a71fba1884099376900fbac9110a6fd3146c3c8a2cc414"} Jul 07 20:06:40.556637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:40.556591 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-gbmbg" podStartSLOduration=1.5565771769999999 podStartE2EDuration="1.556577177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:06:40.555676205 +0000 UTC m=+353.064348781" watchObservedRunningTime="2026-07-07 20:06:40.556577177 +0000 UTC m=+353.065249752" Jul 07 20:06:41.825597 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:41.825560 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:41.951308 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:41.951272 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:44.134791 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.134750 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:44.492438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.492409 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl"] Jul 07 20:06:44.495666 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.495650 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.498671 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.498646 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 20:06:44.498766 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.498701 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 20:06:44.499911 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.499890 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-256hp\"" Jul 07 20:06:44.504692 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.504670 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl"] Jul 07 20:06:44.540832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.540807 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.540963 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.540839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.540963 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.540862 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhtdh\" (UniqueName: \"kubernetes.io/projected/1358901b-53b3-461c-ac8c-af21c21e4224-kube-api-access-dhtdh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.641716 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.641692 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.641948 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.641721 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.641948 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.641743 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhtdh\" (UniqueName: \"kubernetes.io/projected/1358901b-53b3-461c-ac8c-af21c21e4224-kube-api-access-dhtdh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.642108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.642048 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.642172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.642139 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.651931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.651903 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhtdh\" (UniqueName: \"kubernetes.io/projected/1358901b-53b3-461c-ac8c-af21c21e4224-kube-api-access-dhtdh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.804895 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.804822 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:44.937004 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:44.936967 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl"] Jul 07 20:06:44.939345 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:06:44.939320 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1358901b_53b3_461c_ac8c_af21c21e4224.slice/crio-6e417bfd05f462fc535b258a4d783c693231e9b167ce5535ac7c049a5553b343 WatchSource:0}: Error finding container 6e417bfd05f462fc535b258a4d783c693231e9b167ce5535ac7c049a5553b343: Status 404 returned error can't find the container with id 6e417bfd05f462fc535b258a4d783c693231e9b167ce5535ac7c049a5553b343 Jul 07 20:06:45.557481 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:45.557445 2553 generic.go:356] "Generic (PLEG): container finished" podID="1358901b-53b3-461c-ac8c-af21c21e4224" containerID="31dcc95901d2bb72612c6474e6adc6a8cba62d13a7863d28a0296950dcd57c32" exitCode=0 Jul 07 20:06:45.557883 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:45.557534 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" event={"ID":"1358901b-53b3-461c-ac8c-af21c21e4224","Type":"ContainerDied","Data":"31dcc95901d2bb72612c6474e6adc6a8cba62d13a7863d28a0296950dcd57c32"} Jul 07 20:06:45.557883 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:45.557594 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" event={"ID":"1358901b-53b3-461c-ac8c-af21c21e4224","Type":"ContainerStarted","Data":"6e417bfd05f462fc535b258a4d783c693231e9b167ce5535ac7c049a5553b343"} Jul 07 20:06:46.563547 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:46.563513 2553 generic.go:356] "Generic (PLEG): container finished" podID="1358901b-53b3-461c-ac8c-af21c21e4224" containerID="0d4dbc66c9ac5b4979548f486e3538520384e15a332b6b547b1fe6b8ce1eda00" exitCode=0 Jul 07 20:06:46.563933 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:46.563593 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" event={"ID":"1358901b-53b3-461c-ac8c-af21c21e4224","Type":"ContainerDied","Data":"0d4dbc66c9ac5b4979548f486e3538520384e15a332b6b547b1fe6b8ce1eda00"} Jul 07 20:06:47.570078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:47.570047 2553 generic.go:356] "Generic (PLEG): container finished" podID="1358901b-53b3-461c-ac8c-af21c21e4224" containerID="ce9d07dcb55a78ac9b95cc1afdf08e405afb6b460ef8b7aacfce13b3fad5b148" exitCode=0 Jul 07 20:06:47.570446 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:47.570114 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" event={"ID":"1358901b-53b3-461c-ac8c-af21c21e4224","Type":"ContainerDied","Data":"ce9d07dcb55a78ac9b95cc1afdf08e405afb6b460ef8b7aacfce13b3fad5b148"} Jul 07 20:06:48.706072 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.704647 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:48.777623 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.777586 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-bundle\") pod \"1358901b-53b3-461c-ac8c-af21c21e4224\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " Jul 07 20:06:48.777762 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.777646 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-util\") pod \"1358901b-53b3-461c-ac8c-af21c21e4224\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " Jul 07 20:06:48.777762 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.777693 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhtdh\" (UniqueName: \"kubernetes.io/projected/1358901b-53b3-461c-ac8c-af21c21e4224-kube-api-access-dhtdh\") pod \"1358901b-53b3-461c-ac8c-af21c21e4224\" (UID: \"1358901b-53b3-461c-ac8c-af21c21e4224\") " Jul 07 20:06:48.778503 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.778469 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-bundle" (OuterVolumeSpecName: "bundle") pod "1358901b-53b3-461c-ac8c-af21c21e4224" (UID: "1358901b-53b3-461c-ac8c-af21c21e4224"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:06:48.779741 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.779710 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1358901b-53b3-461c-ac8c-af21c21e4224-kube-api-access-dhtdh" (OuterVolumeSpecName: "kube-api-access-dhtdh") pod "1358901b-53b3-461c-ac8c-af21c21e4224" (UID: "1358901b-53b3-461c-ac8c-af21c21e4224"). InnerVolumeSpecName "kube-api-access-dhtdh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:06:48.783377 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.783346 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-util" (OuterVolumeSpecName: "util") pod "1358901b-53b3-461c-ac8c-af21c21e4224" (UID: "1358901b-53b3-461c-ac8c-af21c21e4224"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:06:48.878906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.878841 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhtdh\" (UniqueName: \"kubernetes.io/projected/1358901b-53b3-461c-ac8c-af21c21e4224-kube-api-access-dhtdh\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:48.878906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.878865 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:48.878906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:48.878875 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1358901b-53b3-461c-ac8c-af21c21e4224-util\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:06:49.580818 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:49.580787 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" Jul 07 20:06:49.580818 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:49.580812 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrdfl" event={"ID":"1358901b-53b3-461c-ac8c-af21c21e4224","Type":"ContainerDied","Data":"6e417bfd05f462fc535b258a4d783c693231e9b167ce5535ac7c049a5553b343"} Jul 07 20:06:49.581057 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:49.580864 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e417bfd05f462fc535b258a4d783c693231e9b167ce5535ac7c049a5553b343" Jul 07 20:06:56.444697 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:56.444658 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:56.539537 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:56.539505 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:57.050418 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.050375 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:57.377486 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.377406 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:57.472783 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.472737 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:57.509221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509187 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6"] Jul 07 20:06:57.509532 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509517 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1358901b-53b3-461c-ac8c-af21c21e4224" containerName="util" Jul 07 20:06:57.509587 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509536 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1358901b-53b3-461c-ac8c-af21c21e4224" containerName="util" Jul 07 20:06:57.509587 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509552 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1358901b-53b3-461c-ac8c-af21c21e4224" containerName="pull" Jul 07 20:06:57.509587 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509558 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1358901b-53b3-461c-ac8c-af21c21e4224" containerName="pull" Jul 07 20:06:57.509587 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509564 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1358901b-53b3-461c-ac8c-af21c21e4224" containerName="extract" Jul 07 20:06:57.509587 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509570 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1358901b-53b3-461c-ac8c-af21c21e4224" containerName="extract" Jul 07 20:06:57.509756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.509638 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1358901b-53b3-461c-ac8c-af21c21e4224" containerName="extract" Jul 07 20:06:57.515300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.515279 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.518233 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.518199 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 20:06:57.518358 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.518254 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zjd5h\"" Jul 07 20:06:57.518458 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.518443 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 20:06:57.518610 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.518592 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 20:06:57.519052 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.519035 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 20:06:57.531325 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.531304 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6"] Jul 07 20:06:57.649265 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.649191 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-manifests\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.649265 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.649222 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-webhook-cert\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.649265 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.649247 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.649465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.649352 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-tmp\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.649465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.649420 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcx2z\" (UniqueName: \"kubernetes.io/projected/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-kube-api-access-xcx2z\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.714198 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.714162 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:57.750206 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.750176 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcx2z\" (UniqueName: \"kubernetes.io/projected/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-kube-api-access-xcx2z\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.750345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.750223 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-manifests\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.750345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.750243 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-webhook-cert\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.750345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.750260 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.750345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.750290 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-tmp\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.750631 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.750610 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-manifests\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.752660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.752632 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-tmp\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.752780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.752763 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-webhook-cert\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.752875 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.752858 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.761837 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.761813 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcx2z\" (UniqueName: \"kubernetes.io/projected/aa27f1fb-4473-4141-b9ca-5b75edb99ca8-kube-api-access-xcx2z\") pod \"opendatahub-operator-controller-manager-659495cbcb-w4ct6\" (UID: \"aa27f1fb-4473-4141-b9ca-5b75edb99ca8\") " pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.821912 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.821871 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:57.825808 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.825790 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:06:57.916351 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.916263 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:57.980426 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.980402 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6"] Jul 07 20:06:57.982211 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:06:57.982185 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa27f1fb_4473_4141_b9ca_5b75edb99ca8.slice/crio-646d954374475b15147de0e86e815cb9c892c97e21c9b685b4be1e7af338e769 WatchSource:0}: Error finding container 646d954374475b15147de0e86e815cb9c892c97e21c9b685b4be1e7af338e769: Status 404 returned error can't find the container with id 646d954374475b15147de0e86e815cb9c892c97e21c9b685b4be1e7af338e769 Jul 07 20:06:57.983611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:57.983585 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:58.053160 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:58.053129 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:58.157764 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:58.157730 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:58.254667 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:58.254613 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:06:58.621618 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:06:58.621535 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" event={"ID":"aa27f1fb-4473-4141-b9ca-5b75edb99ca8","Type":"ContainerStarted","Data":"646d954374475b15147de0e86e815cb9c892c97e21c9b685b4be1e7af338e769"} Jul 07 20:07:01.639267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:01.639233 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" event={"ID":"aa27f1fb-4473-4141-b9ca-5b75edb99ca8","Type":"ContainerStarted","Data":"dc9659abc68f9c55d3096ca024af2b3fe31503c05720f9582a4d0ede040ac297"} Jul 07 20:07:02.645227 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:02.645146 2553 generic.go:356] "Generic (PLEG): container finished" podID="aa27f1fb-4473-4141-b9ca-5b75edb99ca8" containerID="dc9659abc68f9c55d3096ca024af2b3fe31503c05720f9582a4d0ede040ac297" exitCode=0 Jul 07 20:07:02.645563 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:02.645234 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" event={"ID":"aa27f1fb-4473-4141-b9ca-5b75edb99ca8","Type":"ContainerDied","Data":"dc9659abc68f9c55d3096ca024af2b3fe31503c05720f9582a4d0ede040ac297"} Jul 07 20:07:03.651712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:03.651631 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" event={"ID":"aa27f1fb-4473-4141-b9ca-5b75edb99ca8","Type":"ContainerStarted","Data":"b5c501c18ff52691d3d10113df25022cfbcc1d8ca4e45faf772c33942dae4893"} Jul 07 20:07:03.651712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:03.651705 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:07:03.683627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:03.683584 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" podStartSLOduration=1.8010972079999998 podStartE2EDuration="6.683568593s" totalImagesPullingTime="3.804595636s" totalInitContainerRuntime="1.077875749s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:06:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:07:03.682027299 +0000 UTC m=+376.190699876" watchObservedRunningTime="2026-07-07 20:07:03.683568593 +0000 UTC m=+376.192241167" Jul 07 20:07:08.595116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:08.595079 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:08.691742 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:08.691703 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:14.658146 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:14.658119 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-659495cbcb-w4ct6" Jul 07 20:07:16.296773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.296728 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:16.381903 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.381870 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:16.397900 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.397870 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd"] Jul 07 20:07:16.401184 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.401168 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.405936 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.405702 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 20:07:16.405936 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.405711 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fjfv5\"" Jul 07 20:07:16.405936 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.405787 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 20:07:16.405936 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.405812 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 20:07:16.406217 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.405984 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 20:07:16.406217 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.406036 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 20:07:16.411846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.411823 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd"] Jul 07 20:07:16.507413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.507365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f815a95f-1826-409e-9b70-ee21022297c7-metrics-cert\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.507585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.507427 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f815a95f-1826-409e-9b70-ee21022297c7-manager-config\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.507585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.507496 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f815a95f-1826-409e-9b70-ee21022297c7-cert\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.507585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.507536 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2hqt\" (UniqueName: \"kubernetes.io/projected/f815a95f-1826-409e-9b70-ee21022297c7-kube-api-access-f2hqt\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.608354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.608273 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f815a95f-1826-409e-9b70-ee21022297c7-metrics-cert\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.608354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.608323 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f815a95f-1826-409e-9b70-ee21022297c7-manager-config\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.608524 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.608358 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f815a95f-1826-409e-9b70-ee21022297c7-cert\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.608524 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.608382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2hqt\" (UniqueName: \"kubernetes.io/projected/f815a95f-1826-409e-9b70-ee21022297c7-kube-api-access-f2hqt\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.609137 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.609115 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f815a95f-1826-409e-9b70-ee21022297c7-manager-config\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.610968 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.610945 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f815a95f-1826-409e-9b70-ee21022297c7-metrics-cert\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.611111 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.610952 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f815a95f-1826-409e-9b70-ee21022297c7-cert\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.618874 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.618851 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2hqt\" (UniqueName: \"kubernetes.io/projected/f815a95f-1826-409e-9b70-ee21022297c7-kube-api-access-f2hqt\") pod \"lws-controller-manager-699bddd6bc-hx5gd\" (UID: \"f815a95f-1826-409e-9b70-ee21022297c7\") " pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.710643 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.710612 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:16.800534 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.800477 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:16.868789 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.868763 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd"] Jul 07 20:07:16.871359 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:07:16.871336 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf815a95f_1826_409e_9b70_ee21022297c7.slice/crio-a215fc1ff5f7c8c6ff7ec18c6fea8c80db3e2aade0e288b4b32eb2af9f1dcb40 WatchSource:0}: Error finding container a215fc1ff5f7c8c6ff7ec18c6fea8c80db3e2aade0e288b4b32eb2af9f1dcb40: Status 404 returned error can't find the container with id a215fc1ff5f7c8c6ff7ec18c6fea8c80db3e2aade0e288b4b32eb2af9f1dcb40 Jul 07 20:07:16.873018 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.872882 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:16.949552 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:16.949514 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:17.237873 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:17.237837 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:17.376267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:17.376241 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:17.715215 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:17.715137 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" event={"ID":"f815a95f-1826-409e-9b70-ee21022297c7","Type":"ContainerStarted","Data":"a215fc1ff5f7c8c6ff7ec18c6fea8c80db3e2aade0e288b4b32eb2af9f1dcb40"} Jul 07 20:07:17.759146 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:17.759106 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:17.863941 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:17.863900 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:17.968090 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:17.967961 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:18.148813 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:18.148768 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:18.722411 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:18.722312 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" event={"ID":"f815a95f-1826-409e-9b70-ee21022297c7","Type":"ContainerStarted","Data":"bc5fa9e5de6b1303d48ddadda81bc952ac0c075f302597859a2f774bea0d2f79"} Jul 07 20:07:18.722841 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:18.722459 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:18.765818 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:18.765765 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" podStartSLOduration=1.190975688 podStartE2EDuration="2.765749647s" totalImagesPullingTime="1.574773959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:07:18.763876239 +0000 UTC m=+391.272548815" watchObservedRunningTime="2026-07-07 20:07:18.765749647 +0000 UTC m=+391.274422222" Jul 07 20:07:18.975580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:18.975481 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:19.135454 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:19.135421 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:19.212728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:19.212693 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:20.320450 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:20.320411 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:25.265496 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.265447 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:25.365487 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.365448 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:25.507901 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.507870 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk"] Jul 07 20:07:25.511307 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.511287 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.514236 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.514216 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 20:07:25.514327 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.514219 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pjh25\"" Jul 07 20:07:25.524929 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.524906 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk"] Jul 07 20:07:25.537589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.537568 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:25.581967 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.581937 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.581986 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.582054 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.582084 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.582114 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.582136 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thf5f\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-kube-api-access-thf5f\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.582163 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.582235 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.582414 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.582275 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.604977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.604949 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v"] Jul 07 20:07:25.608209 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.608194 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.628221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.628197 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v"] Jul 07 20:07:25.683297 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683266 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.683297 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683296 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.683511 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683320 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.683511 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683341 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.683511 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683428 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.683511 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683476 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.683511 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683509 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.683756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683541 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.683756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683583 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.683756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683618 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjmp8\" (UniqueName: \"kubernetes.io/projected/c8753cb8-496d-475b-a348-1099fc543770-kube-api-access-sjmp8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.683756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683648 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thf5f\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-kube-api-access-thf5f\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.683756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683687 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8753cb8-496d-475b-a348-1099fc543770-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.683756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683713 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8753cb8-496d-475b-a348-1099fc543770-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.683756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683748 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.684219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683774 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.684219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683838 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.684219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683864 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.684219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683889 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8753cb8-496d-475b-a348-1099fc543770-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.684219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.683908 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.684436 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.684290 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.684436 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.684340 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.684436 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.684382 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.684436 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.684428 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.685667 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.685647 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.685938 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.685921 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.694231 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.694207 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.694469 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.694451 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thf5f\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-kube-api-access-thf5f\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-glkdk\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.716849 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.716821 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:25.784439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784361 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjmp8\" (UniqueName: \"kubernetes.io/projected/c8753cb8-496d-475b-a348-1099fc543770-kube-api-access-sjmp8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784401 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8753cb8-496d-475b-a348-1099fc543770-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784417 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8753cb8-496d-475b-a348-1099fc543770-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784713 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784442 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784713 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784494 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784713 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784520 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8753cb8-496d-475b-a348-1099fc543770-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784713 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784549 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784713 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784574 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.784713 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784647 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.785024 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784955 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.785024 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.784978 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.785144 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.785093 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.785199 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.785155 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.785336 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.785318 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8753cb8-496d-475b-a348-1099fc543770-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.786783 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.786757 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8753cb8-496d-475b-a348-1099fc543770-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.786870 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.786813 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8753cb8-496d-475b-a348-1099fc543770-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.795637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.795617 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjmp8\" (UniqueName: \"kubernetes.io/projected/c8753cb8-496d-475b-a348-1099fc543770-kube-api-access-sjmp8\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.799312 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.799291 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8753cb8-496d-475b-a348-1099fc543770-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4j7j6v\" (UID: \"c8753cb8-496d-475b-a348-1099fc543770\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.810447 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.810411 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:25.821244 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.821221 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:25.919059 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.918984 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:25.970581 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:25.969862 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk"] Jul 07 20:07:26.066259 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:26.066234 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v"] Jul 07 20:07:26.068638 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:07:26.068610 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8753cb8_496d_475b_a348_1099fc543770.slice/crio-0d4b9bdd5a41b7d6125e8f30d23a3bd1b7fd601427d3c022a41cf5d2f29029cb WatchSource:0}: Error finding container 0d4b9bdd5a41b7d6125e8f30d23a3bd1b7fd601427d3c022a41cf5d2f29029cb: Status 404 returned error can't find the container with id 0d4b9bdd5a41b7d6125e8f30d23a3bd1b7fd601427d3c022a41cf5d2f29029cb Jul 07 20:07:26.760139 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:26.760095 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" event={"ID":"c8753cb8-496d-475b-a348-1099fc543770","Type":"ContainerStarted","Data":"0d4b9bdd5a41b7d6125e8f30d23a3bd1b7fd601427d3c022a41cf5d2f29029cb"} Jul 07 20:07:26.761547 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:26.761518 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" event={"ID":"ad42dd76-b914-43b9-a312-5aa134ef5c9e","Type":"ContainerStarted","Data":"b13c6ecb8cdd68cbb0711961237319b1b37aa967bae54b2825da3c51a53785ab"} Jul 07 20:07:29.730142 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:29.730101 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-699bddd6bc-hx5gd" Jul 07 20:07:29.865923 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:29.865874 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:07:29.866053 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:29.865950 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:07:29.866053 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:29.865976 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:07:29.878505 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:29.878477 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:07:29.878619 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:29.878560 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:07:29.878619 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:29.878590 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:07:30.013611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.013490 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:30.120667 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.120453 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:30.125445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.125416 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh"] Jul 07 20:07:30.129441 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.129419 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.138461 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.138048 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-54gh9\"" Jul 07 20:07:30.138461 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.138303 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 20:07:30.149786 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.149473 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh"] Jul 07 20:07:30.154179 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.153824 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k"] Jul 07 20:07:30.157792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.157769 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.161363 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.161189 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-862s8\"" Jul 07 20:07:30.161363 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.161314 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 20:07:30.173378 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.173350 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k"] Jul 07 20:07:30.203958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.203923 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:30.233402 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.233334 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtnhc\" (UniqueName: \"kubernetes.io/projected/5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b-kube-api-access-dtnhc\") pod \"kserve-controller-manager-d6f4b65b8-mn6xh\" (UID: \"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.233402 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.233369 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6db2109b-9c26-410c-ab24-e9c3f4a1c162-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bqj2k\" (UID: \"6db2109b-9c26-410c-ab24-e9c3f4a1c162\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.233579 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.233446 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77q88\" (UniqueName: \"kubernetes.io/projected/6db2109b-9c26-410c-ab24-e9c3f4a1c162-kube-api-access-77q88\") pod \"llmisvc-controller-manager-777c6bd7d5-bqj2k\" (UID: \"6db2109b-9c26-410c-ab24-e9c3f4a1c162\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.233579 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.233494 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b-cert\") pod \"kserve-controller-manager-d6f4b65b8-mn6xh\" (UID: \"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.336071 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.334547 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtnhc\" (UniqueName: \"kubernetes.io/projected/5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b-kube-api-access-dtnhc\") pod \"kserve-controller-manager-d6f4b65b8-mn6xh\" (UID: \"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.336071 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.334600 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6db2109b-9c26-410c-ab24-e9c3f4a1c162-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bqj2k\" (UID: \"6db2109b-9c26-410c-ab24-e9c3f4a1c162\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.336071 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.334683 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77q88\" (UniqueName: \"kubernetes.io/projected/6db2109b-9c26-410c-ab24-e9c3f4a1c162-kube-api-access-77q88\") pod \"llmisvc-controller-manager-777c6bd7d5-bqj2k\" (UID: \"6db2109b-9c26-410c-ab24-e9c3f4a1c162\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.336071 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.334746 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b-cert\") pod \"kserve-controller-manager-d6f4b65b8-mn6xh\" (UID: \"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.336071 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:07:30.335345 2553 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 07 20:07:30.336071 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:07:30.335414 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6db2109b-9c26-410c-ab24-e9c3f4a1c162-cert podName:6db2109b-9c26-410c-ab24-e9c3f4a1c162 nodeName:}" failed. No retries permitted until 2026-07-07 20:07:30.835391006 +0000 UTC m=+403.344063567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6db2109b-9c26-410c-ab24-e9c3f4a1c162-cert") pod "llmisvc-controller-manager-777c6bd7d5-bqj2k" (UID: "6db2109b-9c26-410c-ab24-e9c3f4a1c162") : secret "llmisvc-webhook-server-cert" not found Jul 07 20:07:30.341012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.338548 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b-cert\") pod \"kserve-controller-manager-d6f4b65b8-mn6xh\" (UID: \"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.350978 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.350925 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtnhc\" (UniqueName: \"kubernetes.io/projected/5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b-kube-api-access-dtnhc\") pod \"kserve-controller-manager-d6f4b65b8-mn6xh\" (UID: \"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.354666 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.354639 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77q88\" (UniqueName: \"kubernetes.io/projected/6db2109b-9c26-410c-ab24-e9c3f4a1c162-kube-api-access-77q88\") pod \"llmisvc-controller-manager-777c6bd7d5-bqj2k\" (UID: \"6db2109b-9c26-410c-ab24-e9c3f4a1c162\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.443246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.443215 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:30.477457 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.477424 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:30.587220 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.587194 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh"] Jul 07 20:07:30.588689 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:07:30.588662 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bbd2b4a_8d41_4849_bfe1_bbe11dd6616b.slice/crio-7c6723e7a12fec7875f114a277ecf7d208d06fedf9a6cd6006891e9c46c1e131 WatchSource:0}: Error finding container 7c6723e7a12fec7875f114a277ecf7d208d06fedf9a6cd6006891e9c46c1e131: Status 404 returned error can't find the container with id 7c6723e7a12fec7875f114a277ecf7d208d06fedf9a6cd6006891e9c46c1e131 Jul 07 20:07:30.784680 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.784642 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" event={"ID":"ad42dd76-b914-43b9-a312-5aa134ef5c9e","Type":"ContainerStarted","Data":"f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344"} Jul 07 20:07:30.786309 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.786281 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" event={"ID":"c8753cb8-496d-475b-a348-1099fc543770","Type":"ContainerStarted","Data":"301c6cb9ebe768506ddbd267d5e78e88b13c7d9510988f19cf17a46d287a4092"} Jul 07 20:07:30.787495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.787472 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" event={"ID":"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b","Type":"ContainerStarted","Data":"7c6723e7a12fec7875f114a277ecf7d208d06fedf9a6cd6006891e9c46c1e131"} Jul 07 20:07:30.811321 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.811272 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" podStartSLOduration=1.914891248 podStartE2EDuration="5.811258482s" totalImagesPullingTime="3.896367234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:07:30.809892165 +0000 UTC m=+403.318564770" watchObservedRunningTime="2026-07-07 20:07:30.811258482 +0000 UTC m=+403.319931057" Jul 07 20:07:30.822356 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.822324 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:30.823628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.823601 2553 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 20:07:30.823736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.823643 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 20:07:30.832648 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.832593 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" podStartSLOduration=2.03758624 podStartE2EDuration="5.832568613s" totalImagesPullingTime="3.794982373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:07:30.829859616 +0000 UTC m=+403.338532193" watchObservedRunningTime="2026-07-07 20:07:30.832568613 +0000 UTC m=+403.341241189" Jul 07 20:07:30.839888 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.839563 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6db2109b-9c26-410c-ab24-e9c3f4a1c162-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bqj2k\" (UID: \"6db2109b-9c26-410c-ab24-e9c3f4a1c162\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.841856 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.841830 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6db2109b-9c26-410c-ab24-e9c3f4a1c162-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bqj2k\" (UID: \"6db2109b-9c26-410c-ab24-e9c3f4a1c162\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:30.919939 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.919901 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:30.925098 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:30.925076 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:31.071143 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.071112 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:31.207729 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.207702 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k"] Jul 07 20:07:31.209715 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:07:31.209691 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6db2109b_9c26_410c_ab24_e9c3f4a1c162.slice/crio-6c1ba078c18d45cfa6327577b66082dd2d789903a3fc1731b33c427ecde5d5dc WatchSource:0}: Error finding container 6c1ba078c18d45cfa6327577b66082dd2d789903a3fc1731b33c427ecde5d5dc: Status 404 returned error can't find the container with id 6c1ba078c18d45cfa6327577b66082dd2d789903a3fc1731b33c427ecde5d5dc Jul 07 20:07:31.409582 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.409497 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:31.507526 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.507488 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:31.606906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.606871 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:07:31.805083 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.804981 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" event={"ID":"6db2109b-9c26-410c-ab24-e9c3f4a1c162","Type":"ContainerStarted","Data":"6c1ba078c18d45cfa6327577b66082dd2d789903a3fc1731b33c427ecde5d5dc"} Jul 07 20:07:31.805543 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.805473 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:31.806678 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.806628 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4j7j6v" Jul 07 20:07:31.822130 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.822101 2553 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 20:07:31.822289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.822157 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 20:07:31.878971 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:31.878936 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk"] Jul 07 20:07:32.821796 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:32.821762 2553 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 20:07:32.822269 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:32.821821 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 20:07:33.816010 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:33.815946 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerName="istio-proxy" containerID="cri-o://f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344" gracePeriod=30 Jul 07 20:07:34.824017 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:34.823963 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" event={"ID":"5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b","Type":"ContainerStarted","Data":"9760d882741043b0a051e6165c3ef43bfb8ce59bf502dd4edcc801bf87717f4f"} Jul 07 20:07:34.824431 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:34.824082 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:07:34.847320 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:34.847219 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" podStartSLOduration=1.05005931 podStartE2EDuration="4.847199655s" totalImagesPullingTime="3.797140345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:07:34.843250071 +0000 UTC m=+407.351922647" watchObservedRunningTime="2026-07-07 20:07:34.847199655 +0000 UTC m=+407.355872232" Jul 07 20:07:35.829487 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:35.829450 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" event={"ID":"6db2109b-9c26-410c-ab24-e9c3f4a1c162","Type":"ContainerStarted","Data":"073be7a1747df54938e8cd79cee036b642fc38f83e2c6d74b2aa1546b670c461"} Jul 07 20:07:35.829865 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:35.829549 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:07:35.850100 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:35.850052 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" podStartSLOduration=2.068351709 podStartE2EDuration="5.850039861s" totalImagesPullingTime="3.781688152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:07:35.847959042 +0000 UTC m=+408.356631614" watchObservedRunningTime="2026-07-07 20:07:35.850039861 +0000 UTC m=+408.358712435" Jul 07 20:07:39.067488 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.067463 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:39.113693 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113621 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-credential-socket\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.113833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113695 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-socket\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.113833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113728 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thf5f\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-kube-api-access-thf5f\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.113833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113757 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-token\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.113833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113814 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-envoy\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.114043 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113843 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istiod-ca-cert\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.114043 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113871 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-podinfo\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.114043 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113899 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:07:39.114043 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113926 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-certs\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.114043 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.113954 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-data\") pod \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\" (UID: \"ad42dd76-b914-43b9-a312-5aa134ef5c9e\") " Jul 07 20:07:39.114284 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114069 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:07:39.114338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114238 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:07:39.114338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114314 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:07:39.114763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114568 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:07:39.114763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114664 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.114763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114683 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.114763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114699 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.114763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114713 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.114763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.114728 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.116712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.116679 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:07:39.116712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.116682 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:07:39.116888 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.116684 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-kube-api-access-thf5f" (OuterVolumeSpecName: "kube-api-access-thf5f") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "kube-api-access-thf5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:07:39.116888 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.116681 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad42dd76-b914-43b9-a312-5aa134ef5c9e" (UID: "ad42dd76-b914-43b9-a312-5aa134ef5c9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:07:39.215258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.215231 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thf5f\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-kube-api-access-thf5f\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.215258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.215255 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.215258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.215265 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.215540 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.215275 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad42dd76-b914-43b9-a312-5aa134ef5c9e-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:07:39.850111 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.850077 2553 generic.go:356] "Generic (PLEG): container finished" podID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerID="f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344" exitCode=0 Jul 07 20:07:39.850281 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.850125 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" event={"ID":"ad42dd76-b914-43b9-a312-5aa134ef5c9e","Type":"ContainerDied","Data":"f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344"} Jul 07 20:07:39.850281 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.850138 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" Jul 07 20:07:39.850281 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.850195 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk" event={"ID":"ad42dd76-b914-43b9-a312-5aa134ef5c9e","Type":"ContainerDied","Data":"b13c6ecb8cdd68cbb0711961237319b1b37aa967bae54b2825da3c51a53785ab"} Jul 07 20:07:39.850281 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.850223 2553 scope.go:117] "RemoveContainer" containerID="f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344" Jul 07 20:07:39.859732 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.859715 2553 scope.go:117] "RemoveContainer" containerID="f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344" Jul 07 20:07:39.859984 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:07:39.859964 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344\": container with ID starting with f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344 not found: ID does not exist" containerID="f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344" Jul 07 20:07:39.860077 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.860006 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344"} err="failed to get container status \"f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344\": rpc error: code = NotFound desc = could not find container \"f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344\": container with ID starting with f73567ae20bd051f4543cb58a962633c61d47e271f7df79ce5efb0dd6cee1344 not found: ID does not exist" Jul 07 20:07:39.879529 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.879501 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk"] Jul 07 20:07:39.881928 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:39.881907 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-glkdk"] Jul 07 20:07:40.030117 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:07:40.030087 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" path="/var/lib/kubelet/pods/ad42dd76-b914-43b9-a312-5aa134ef5c9e/volumes" Jul 07 20:08:05.834854 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:05.834819 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mn6xh" Jul 07 20:08:06.837106 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:06.837075 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bqj2k" Jul 07 20:08:18.506271 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:18.506232 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:18.613438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:18.613397 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:18.752117 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:18.752078 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:34.051116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.051080 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq"] Jul 07 20:08:34.051491 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.051417 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerName="istio-proxy" Jul 07 20:08:34.051491 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.051428 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerName="istio-proxy" Jul 07 20:08:34.051563 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.051504 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad42dd76-b914-43b9-a312-5aa134ef5c9e" containerName="istio-proxy" Jul 07 20:08:34.054626 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.054609 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.057763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.057734 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 20:08:34.057909 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.057789 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 20:08:34.058943 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.058917 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zqrbq\"" Jul 07 20:08:34.064167 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.064141 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq"] Jul 07 20:08:34.173694 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.173656 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.173694 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.173703 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg6n2\" (UniqueName: \"kubernetes.io/projected/cfc749c5-50dd-4f05-a839-b154db80c33d-kube-api-access-rg6n2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.173904 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.173810 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.274964 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.274919 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.275186 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.274976 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg6n2\" (UniqueName: \"kubernetes.io/projected/cfc749c5-50dd-4f05-a839-b154db80c33d-kube-api-access-rg6n2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.275186 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.275079 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.275503 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.275344 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.275503 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.275440 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.287358 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.287328 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg6n2\" (UniqueName: \"kubernetes.io/projected/cfc749c5-50dd-4f05-a839-b154db80c33d-kube-api-access-rg6n2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.364571 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.364473 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:34.501968 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.501939 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq"] Jul 07 20:08:34.504603 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:08:34.504574 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc749c5_50dd_4f05_a839_b154db80c33d.slice/crio-4a01589951bc10e0b83caca032db6565b782d587174fecf0584a57d0d9bd614b WatchSource:0}: Error finding container 4a01589951bc10e0b83caca032db6565b782d587174fecf0584a57d0d9bd614b: Status 404 returned error can't find the container with id 4a01589951bc10e0b83caca032db6565b782d587174fecf0584a57d0d9bd614b Jul 07 20:08:34.651555 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.651521 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9"] Jul 07 20:08:34.655055 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.655031 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.664498 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.664465 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9"] Jul 07 20:08:34.779068 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.779037 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.779224 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.779096 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.779224 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.779201 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrp2m\" (UniqueName: \"kubernetes.io/projected/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-kube-api-access-rrp2m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.880449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.880366 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.880581 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.880478 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrp2m\" (UniqueName: \"kubernetes.io/projected/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-kube-api-access-rrp2m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.880581 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.880525 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.880657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.880629 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.880806 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.880779 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.890621 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.890591 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrp2m\" (UniqueName: \"kubernetes.io/projected/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-kube-api-access-rrp2m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:34.971365 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:34.971329 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:35.085012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.084046 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh"] Jul 07 20:08:35.099342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.095419 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh"] Jul 07 20:08:35.099342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.095562 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.120246 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.120211 2553 generic.go:356] "Generic (PLEG): container finished" podID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerID="34d392e60ba722c40922f574d0ec6c9fa04e20c05c271dbc67253b484cc974df" exitCode=0 Jul 07 20:08:35.120534 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.120256 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" event={"ID":"cfc749c5-50dd-4f05-a839-b154db80c33d","Type":"ContainerDied","Data":"34d392e60ba722c40922f574d0ec6c9fa04e20c05c271dbc67253b484cc974df"} Jul 07 20:08:35.120647 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.120578 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" event={"ID":"cfc749c5-50dd-4f05-a839-b154db80c33d","Type":"ContainerStarted","Data":"4a01589951bc10e0b83caca032db6565b782d587174fecf0584a57d0d9bd614b"} Jul 07 20:08:35.123137 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.123055 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9"] Jul 07 20:08:35.125807 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:08:35.125784 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10ecf9b_c292_435a_9f4b_e7f61d38b8ce.slice/crio-4e6fbbc03f61fe2fc3558b95e7a2781ee38bf2c990fa0ed889524778be57b9e3 WatchSource:0}: Error finding container 4e6fbbc03f61fe2fc3558b95e7a2781ee38bf2c990fa0ed889524778be57b9e3: Status 404 returned error can't find the container with id 4e6fbbc03f61fe2fc3558b95e7a2781ee38bf2c990fa0ed889524778be57b9e3 Jul 07 20:08:35.193982 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.193951 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h59c\" (UniqueName: \"kubernetes.io/projected/018ba219-08fc-46fb-91ec-c0dd64d0dda3-kube-api-access-7h59c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.194144 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.194016 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.194144 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.194084 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.295271 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.295242 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.295363 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.295302 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h59c\" (UniqueName: \"kubernetes.io/projected/018ba219-08fc-46fb-91ec-c0dd64d0dda3-kube-api-access-7h59c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.295363 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.295334 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.295718 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.295689 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.295827 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.295702 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.305694 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.305667 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h59c\" (UniqueName: \"kubernetes.io/projected/018ba219-08fc-46fb-91ec-c0dd64d0dda3-kube-api-access-7h59c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.410044 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.409931 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:35.446829 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.446799 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc"] Jul 07 20:08:35.451422 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.451395 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.460082 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.460053 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc"] Jul 07 20:08:35.551930 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.551908 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh"] Jul 07 20:08:35.553984 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:08:35.553962 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018ba219_08fc_46fb_91ec_c0dd64d0dda3.slice/crio-bbe49ceb29eebfb53272f20ecb0d79e1a97f7286ac48ce78911599553a9a9839 WatchSource:0}: Error finding container bbe49ceb29eebfb53272f20ecb0d79e1a97f7286ac48ce78911599553a9a9839: Status 404 returned error can't find the container with id bbe49ceb29eebfb53272f20ecb0d79e1a97f7286ac48ce78911599553a9a9839 Jul 07 20:08:35.598311 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.598285 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.598427 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.598378 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.598477 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.598449 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnjmn\" (UniqueName: \"kubernetes.io/projected/748bc641-827d-4e44-86a5-0c5685676f1a-kube-api-access-bnjmn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.699554 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.699464 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.699554 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.699529 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnjmn\" (UniqueName: \"kubernetes.io/projected/748bc641-827d-4e44-86a5-0c5685676f1a-kube-api-access-bnjmn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.699791 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.699563 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.699941 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.699916 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.700046 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.699924 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.709640 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.709613 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnjmn\" (UniqueName: \"kubernetes.io/projected/748bc641-827d-4e44-86a5-0c5685676f1a-kube-api-access-bnjmn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.765621 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.765583 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:35.904203 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:35.904180 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc"] Jul 07 20:08:35.906165 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:08:35.906137 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748bc641_827d_4e44_86a5_0c5685676f1a.slice/crio-954587bcc756d43e88c5e1d1605a807788edd5ff92e743219c033843af290d31 WatchSource:0}: Error finding container 954587bcc756d43e88c5e1d1605a807788edd5ff92e743219c033843af290d31: Status 404 returned error can't find the container with id 954587bcc756d43e88c5e1d1605a807788edd5ff92e743219c033843af290d31 Jul 07 20:08:36.127057 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.127023 2553 generic.go:356] "Generic (PLEG): container finished" podID="748bc641-827d-4e44-86a5-0c5685676f1a" containerID="269944b3cff492ecc9df8ec46144b80d196550858b3a237a5f443d8d087b4d8a" exitCode=0 Jul 07 20:08:36.127439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.127100 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" event={"ID":"748bc641-827d-4e44-86a5-0c5685676f1a","Type":"ContainerDied","Data":"269944b3cff492ecc9df8ec46144b80d196550858b3a237a5f443d8d087b4d8a"} Jul 07 20:08:36.127439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.127163 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" event={"ID":"748bc641-827d-4e44-86a5-0c5685676f1a","Type":"ContainerStarted","Data":"954587bcc756d43e88c5e1d1605a807788edd5ff92e743219c033843af290d31"} Jul 07 20:08:36.128668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.128644 2553 generic.go:356] "Generic (PLEG): container finished" podID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerID="34d394536158b278714379519b15adf22dae0195008b91c7fc6bdabc069643f7" exitCode=0 Jul 07 20:08:36.128756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.128724 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" event={"ID":"018ba219-08fc-46fb-91ec-c0dd64d0dda3","Type":"ContainerDied","Data":"34d394536158b278714379519b15adf22dae0195008b91c7fc6bdabc069643f7"} Jul 07 20:08:36.128800 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.128770 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" event={"ID":"018ba219-08fc-46fb-91ec-c0dd64d0dda3","Type":"ContainerStarted","Data":"bbe49ceb29eebfb53272f20ecb0d79e1a97f7286ac48ce78911599553a9a9839"} Jul 07 20:08:36.130373 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.130347 2553 generic.go:356] "Generic (PLEG): container finished" podID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerID="3afe776c2598abb109d811b04979a4faeef006b5f0a0e2f3611f9e0896275fa0" exitCode=0 Jul 07 20:08:36.130480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.130386 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" event={"ID":"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce","Type":"ContainerDied","Data":"3afe776c2598abb109d811b04979a4faeef006b5f0a0e2f3611f9e0896275fa0"} Jul 07 20:08:36.130480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:36.130421 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" event={"ID":"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce","Type":"ContainerStarted","Data":"4e6fbbc03f61fe2fc3558b95e7a2781ee38bf2c990fa0ed889524778be57b9e3"} Jul 07 20:08:37.140163 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:37.140134 2553 generic.go:356] "Generic (PLEG): container finished" podID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerID="87dba41ae70c6be13722929f6799618d1f011b98bebddefb03b6ec6526226cd5" exitCode=0 Jul 07 20:08:37.140580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:37.140190 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" event={"ID":"cfc749c5-50dd-4f05-a839-b154db80c33d","Type":"ContainerDied","Data":"87dba41ae70c6be13722929f6799618d1f011b98bebddefb03b6ec6526226cd5"} Jul 07 20:08:38.152877 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.152844 2553 generic.go:356] "Generic (PLEG): container finished" podID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerID="84602684972b6bbd2fc3b0e437c6c2917989d9526efffe049e0580c212e501f6" exitCode=0 Jul 07 20:08:38.153365 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.152934 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" event={"ID":"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce","Type":"ContainerDied","Data":"84602684972b6bbd2fc3b0e437c6c2917989d9526efffe049e0580c212e501f6"} Jul 07 20:08:38.154885 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.154861 2553 generic.go:356] "Generic (PLEG): container finished" podID="748bc641-827d-4e44-86a5-0c5685676f1a" containerID="e53e5a8bb46451ccc2ab5b1eb17ab0493539d430ded57ea2d7e7ff754749f849" exitCode=0 Jul 07 20:08:38.155018 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.154934 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" event={"ID":"748bc641-827d-4e44-86a5-0c5685676f1a","Type":"ContainerDied","Data":"e53e5a8bb46451ccc2ab5b1eb17ab0493539d430ded57ea2d7e7ff754749f849"} Jul 07 20:08:38.156884 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.156860 2553 generic.go:356] "Generic (PLEG): container finished" podID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerID="bd3552a1529ce35d50ddce55abb69fadabffb47d613e7e051ce71173c3dfee01" exitCode=0 Jul 07 20:08:38.156977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.156938 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" event={"ID":"018ba219-08fc-46fb-91ec-c0dd64d0dda3","Type":"ContainerDied","Data":"bd3552a1529ce35d50ddce55abb69fadabffb47d613e7e051ce71173c3dfee01"} Jul 07 20:08:38.159291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.159271 2553 generic.go:356] "Generic (PLEG): container finished" podID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerID="27ea6839561643015d42d6762331924d3980056da3f06f417b62b1cac302069f" exitCode=0 Jul 07 20:08:38.159390 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:38.159334 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" event={"ID":"cfc749c5-50dd-4f05-a839-b154db80c33d","Type":"ContainerDied","Data":"27ea6839561643015d42d6762331924d3980056da3f06f417b62b1cac302069f"} Jul 07 20:08:39.166050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.166019 2553 generic.go:356] "Generic (PLEG): container finished" podID="748bc641-827d-4e44-86a5-0c5685676f1a" containerID="6b42d7bbc00456860da858e02a9076f1c23123466a9d301d555072fe5e07eac9" exitCode=0 Jul 07 20:08:39.166471 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.166112 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" event={"ID":"748bc641-827d-4e44-86a5-0c5685676f1a","Type":"ContainerDied","Data":"6b42d7bbc00456860da858e02a9076f1c23123466a9d301d555072fe5e07eac9"} Jul 07 20:08:39.168094 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.168071 2553 generic.go:356] "Generic (PLEG): container finished" podID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerID="b6bd83aac904ae7020ef289a577c1222228a5e5cbbf017062f8189a81457d55f" exitCode=0 Jul 07 20:08:39.168163 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.168145 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" event={"ID":"018ba219-08fc-46fb-91ec-c0dd64d0dda3","Type":"ContainerDied","Data":"b6bd83aac904ae7020ef289a577c1222228a5e5cbbf017062f8189a81457d55f"} Jul 07 20:08:39.170105 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.170082 2553 generic.go:356] "Generic (PLEG): container finished" podID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerID="1f40ea10199e63c15bc26b40bed7ee13118d792e5a9fa338c08a52eec8a77efd" exitCode=0 Jul 07 20:08:39.170206 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.170108 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" event={"ID":"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce","Type":"ContainerDied","Data":"1f40ea10199e63c15bc26b40bed7ee13118d792e5a9fa338c08a52eec8a77efd"} Jul 07 20:08:39.302051 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.302024 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:39.430378 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.430287 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg6n2\" (UniqueName: \"kubernetes.io/projected/cfc749c5-50dd-4f05-a839-b154db80c33d-kube-api-access-rg6n2\") pod \"cfc749c5-50dd-4f05-a839-b154db80c33d\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " Jul 07 20:08:39.430378 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.430349 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-util\") pod \"cfc749c5-50dd-4f05-a839-b154db80c33d\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " Jul 07 20:08:39.430378 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.430381 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-bundle\") pod \"cfc749c5-50dd-4f05-a839-b154db80c33d\" (UID: \"cfc749c5-50dd-4f05-a839-b154db80c33d\") " Jul 07 20:08:39.431050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.431015 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-bundle" (OuterVolumeSpecName: "bundle") pod "cfc749c5-50dd-4f05-a839-b154db80c33d" (UID: "cfc749c5-50dd-4f05-a839-b154db80c33d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:39.432588 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.432548 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfc749c5-50dd-4f05-a839-b154db80c33d-kube-api-access-rg6n2" (OuterVolumeSpecName: "kube-api-access-rg6n2") pod "cfc749c5-50dd-4f05-a839-b154db80c33d" (UID: "cfc749c5-50dd-4f05-a839-b154db80c33d"). InnerVolumeSpecName "kube-api-access-rg6n2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:08:39.436267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.436230 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-util" (OuterVolumeSpecName: "util") pod "cfc749c5-50dd-4f05-a839-b154db80c33d" (UID: "cfc749c5-50dd-4f05-a839-b154db80c33d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:39.531391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.531347 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-util\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:39.531391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.531383 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cfc749c5-50dd-4f05-a839-b154db80c33d-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:39.531391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:39.531392 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg6n2\" (UniqueName: \"kubernetes.io/projected/cfc749c5-50dd-4f05-a839-b154db80c33d-kube-api-access-rg6n2\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.176760 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.176730 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" Jul 07 20:08:40.177267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.176768 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq" event={"ID":"cfc749c5-50dd-4f05-a839-b154db80c33d","Type":"ContainerDied","Data":"4a01589951bc10e0b83caca032db6565b782d587174fecf0584a57d0d9bd614b"} Jul 07 20:08:40.177267 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.176823 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a01589951bc10e0b83caca032db6565b782d587174fecf0584a57d0d9bd614b" Jul 07 20:08:40.319277 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.319244 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:40.356136 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.356103 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:40.371025 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.371002 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:40.438999 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.438957 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h59c\" (UniqueName: \"kubernetes.io/projected/018ba219-08fc-46fb-91ec-c0dd64d0dda3-kube-api-access-7h59c\") pod \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " Jul 07 20:08:40.439145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439029 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-util\") pod \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " Jul 07 20:08:40.439145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439053 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnjmn\" (UniqueName: \"kubernetes.io/projected/748bc641-827d-4e44-86a5-0c5685676f1a-kube-api-access-bnjmn\") pod \"748bc641-827d-4e44-86a5-0c5685676f1a\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " Jul 07 20:08:40.439145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439081 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-util\") pod \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " Jul 07 20:08:40.439145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439113 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-bundle\") pod \"748bc641-827d-4e44-86a5-0c5685676f1a\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " Jul 07 20:08:40.439374 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439153 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrp2m\" (UniqueName: \"kubernetes.io/projected/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-kube-api-access-rrp2m\") pod \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " Jul 07 20:08:40.439374 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439203 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-bundle\") pod \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\" (UID: \"018ba219-08fc-46fb-91ec-c0dd64d0dda3\") " Jul 07 20:08:40.439374 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439245 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-bundle\") pod \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\" (UID: \"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce\") " Jul 07 20:08:40.439374 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.439274 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-util\") pod \"748bc641-827d-4e44-86a5-0c5685676f1a\" (UID: \"748bc641-827d-4e44-86a5-0c5685676f1a\") " Jul 07 20:08:40.441475 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.440061 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-bundle" (OuterVolumeSpecName: "bundle") pod "018ba219-08fc-46fb-91ec-c0dd64d0dda3" (UID: "018ba219-08fc-46fb-91ec-c0dd64d0dda3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:40.441475 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.440465 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-bundle" (OuterVolumeSpecName: "bundle") pod "e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" (UID: "e10ecf9b-c292-435a-9f4b-e7f61d38b8ce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:40.441475 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.440553 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-bundle" (OuterVolumeSpecName: "bundle") pod "748bc641-827d-4e44-86a5-0c5685676f1a" (UID: "748bc641-827d-4e44-86a5-0c5685676f1a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:40.442186 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.442160 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018ba219-08fc-46fb-91ec-c0dd64d0dda3-kube-api-access-7h59c" (OuterVolumeSpecName: "kube-api-access-7h59c") pod "018ba219-08fc-46fb-91ec-c0dd64d0dda3" (UID: "018ba219-08fc-46fb-91ec-c0dd64d0dda3"). InnerVolumeSpecName "kube-api-access-7h59c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:08:40.442285 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.442210 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/748bc641-827d-4e44-86a5-0c5685676f1a-kube-api-access-bnjmn" (OuterVolumeSpecName: "kube-api-access-bnjmn") pod "748bc641-827d-4e44-86a5-0c5685676f1a" (UID: "748bc641-827d-4e44-86a5-0c5685676f1a"). InnerVolumeSpecName "kube-api-access-bnjmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:08:40.442458 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.442428 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-kube-api-access-rrp2m" (OuterVolumeSpecName: "kube-api-access-rrp2m") pod "e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" (UID: "e10ecf9b-c292-435a-9f4b-e7f61d38b8ce"). InnerVolumeSpecName "kube-api-access-rrp2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:08:40.445081 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.445056 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-util" (OuterVolumeSpecName: "util") pod "e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" (UID: "e10ecf9b-c292-435a-9f4b-e7f61d38b8ce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:40.446366 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.446341 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-util" (OuterVolumeSpecName: "util") pod "018ba219-08fc-46fb-91ec-c0dd64d0dda3" (UID: "018ba219-08fc-46fb-91ec-c0dd64d0dda3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:40.447387 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.447362 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-util" (OuterVolumeSpecName: "util") pod "748bc641-827d-4e44-86a5-0c5685676f1a" (UID: "748bc641-827d-4e44-86a5-0c5685676f1a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540255 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h59c\" (UniqueName: \"kubernetes.io/projected/018ba219-08fc-46fb-91ec-c0dd64d0dda3-kube-api-access-7h59c\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540286 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-util\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540297 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnjmn\" (UniqueName: \"kubernetes.io/projected/748bc641-827d-4e44-86a5-0c5685676f1a-kube-api-access-bnjmn\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540307 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-util\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540317 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540325 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrp2m\" (UniqueName: \"kubernetes.io/projected/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-kube-api-access-rrp2m\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540333 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/018ba219-08fc-46fb-91ec-c0dd64d0dda3-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540338 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540341 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10ecf9b-c292-435a-9f4b-e7f61d38b8ce-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:40.540745 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:40.540350 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748bc641-827d-4e44-86a5-0c5685676f1a-util\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:08:41.185830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.185802 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" Jul 07 20:08:41.186283 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.185805 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc" event={"ID":"748bc641-827d-4e44-86a5-0c5685676f1a","Type":"ContainerDied","Data":"954587bcc756d43e88c5e1d1605a807788edd5ff92e743219c033843af290d31"} Jul 07 20:08:41.186283 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.185928 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="954587bcc756d43e88c5e1d1605a807788edd5ff92e743219c033843af290d31" Jul 07 20:08:41.187607 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.187584 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" Jul 07 20:08:41.187746 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.187601 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh" event={"ID":"018ba219-08fc-46fb-91ec-c0dd64d0dda3","Type":"ContainerDied","Data":"bbe49ceb29eebfb53272f20ecb0d79e1a97f7286ac48ce78911599553a9a9839"} Jul 07 20:08:41.187746 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.187641 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbe49ceb29eebfb53272f20ecb0d79e1a97f7286ac48ce78911599553a9a9839" Jul 07 20:08:41.189665 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.189644 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" Jul 07 20:08:41.189773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.189659 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9" event={"ID":"e10ecf9b-c292-435a-9f4b-e7f61d38b8ce","Type":"ContainerDied","Data":"4e6fbbc03f61fe2fc3558b95e7a2781ee38bf2c990fa0ed889524778be57b9e3"} Jul 07 20:08:41.189773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:41.189700 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e6fbbc03f61fe2fc3558b95e7a2781ee38bf2c990fa0ed889524778be57b9e3" Jul 07 20:08:47.471833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:47.471789 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:50.828120 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:50.828081 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:51.637549 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:51.637515 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:54.216594 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.216551 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs"] Jul 07 20:08:54.217091 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217051 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerName="extract" Jul 07 20:08:54.217091 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217070 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerName="extract" Jul 07 20:08:54.217091 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217082 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748bc641-827d-4e44-86a5-0c5685676f1a" containerName="extract" Jul 07 20:08:54.217091 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217090 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="748bc641-827d-4e44-86a5-0c5685676f1a" containerName="extract" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217106 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217114 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217127 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748bc641-827d-4e44-86a5-0c5685676f1a" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217136 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="748bc641-827d-4e44-86a5-0c5685676f1a" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217158 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217166 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217176 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748bc641-827d-4e44-86a5-0c5685676f1a" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217183 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="748bc641-827d-4e44-86a5-0c5685676f1a" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217193 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217201 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217214 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerName="extract" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217223 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerName="extract" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217233 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217242 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerName="util" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217261 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217269 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217278 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217285 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerName="pull" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217295 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerName="extract" Jul 07 20:08:54.217323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217303 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerName="extract" Jul 07 20:08:54.218245 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217390 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfc749c5-50dd-4f05-a839-b154db80c33d" containerName="extract" Jul 07 20:08:54.218245 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217405 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e10ecf9b-c292-435a-9f4b-e7f61d38b8ce" containerName="extract" Jul 07 20:08:54.218245 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217415 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="018ba219-08fc-46fb-91ec-c0dd64d0dda3" containerName="extract" Jul 07 20:08:54.218245 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.217425 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="748bc641-827d-4e44-86a5-0c5685676f1a" containerName="extract" Jul 07 20:08:54.220575 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.220549 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" Jul 07 20:08:54.223547 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.223529 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fcq65\"" Jul 07 20:08:54.223636 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.223558 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 20:08:54.224957 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.224942 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 20:08:54.232141 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.232120 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs"] Jul 07 20:08:54.367908 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.367860 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7k46\" (UniqueName: \"kubernetes.io/projected/00b8a46f-cb14-4201-b20a-0ce726199312-kube-api-access-q7k46\") pod \"authorino-operator-75cc8d5cb7-qn5fs\" (UID: \"00b8a46f-cb14-4201-b20a-0ce726199312\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" Jul 07 20:08:54.469092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.468979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7k46\" (UniqueName: \"kubernetes.io/projected/00b8a46f-cb14-4201-b20a-0ce726199312-kube-api-access-q7k46\") pod \"authorino-operator-75cc8d5cb7-qn5fs\" (UID: \"00b8a46f-cb14-4201-b20a-0ce726199312\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" Jul 07 20:08:54.482570 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.482552 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7k46\" (UniqueName: \"kubernetes.io/projected/00b8a46f-cb14-4201-b20a-0ce726199312-kube-api-access-q7k46\") pod \"authorino-operator-75cc8d5cb7-qn5fs\" (UID: \"00b8a46f-cb14-4201-b20a-0ce726199312\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" Jul 07 20:08:54.531892 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.531868 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" Jul 07 20:08:54.672736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.672711 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs"] Jul 07 20:08:54.674893 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:08:54.674860 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b8a46f_cb14_4201_b20a_0ce726199312.slice/crio-f3bf4d6d5b68c8b571e6b254eda99ec9ce7afd4a01f4c37f969141be8a263441 WatchSource:0}: Error finding container f3bf4d6d5b68c8b571e6b254eda99ec9ce7afd4a01f4c37f969141be8a263441: Status 404 returned error can't find the container with id f3bf4d6d5b68c8b571e6b254eda99ec9ce7afd4a01f4c37f969141be8a263441 Jul 07 20:08:54.896454 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:54.896422 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:55.017929 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:55.017894 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:55.228841 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:55.228810 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:55.264681 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:55.264641 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" event={"ID":"00b8a46f-cb14-4201-b20a-0ce726199312","Type":"ContainerStarted","Data":"f3bf4d6d5b68c8b571e6b254eda99ec9ce7afd4a01f4c37f969141be8a263441"} Jul 07 20:08:56.848923 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:56.848888 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:58.026306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:58.026260 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:58.138301 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:58.138267 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:08:58.282337 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:58.282257 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" event={"ID":"00b8a46f-cb14-4201-b20a-0ce726199312","Type":"ContainerStarted","Data":"6f35aaf8cc5ee666d50c6d95e740ff7fbd91448e4d3e5b4de01dd66a016dd8a4"} Jul 07 20:08:58.282498 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:58.282399 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" Jul 07 20:08:58.321367 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:58.321317 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" podStartSLOduration=1.5525009920000001 podStartE2EDuration="4.321302501s" totalImagesPullingTime="2.768801509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:08:58.321149914 +0000 UTC m=+490.829822488" watchObservedRunningTime="2026-07-07 20:08:58.321302501 +0000 UTC m=+490.829975075" Jul 07 20:08:59.866343 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:08:59.866304 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:00.409890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.409854 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bbc4f5cf6-wgc57"] Jul 07 20:09:00.413706 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.413680 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.425227 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.425204 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbc4f5cf6-wgc57"] Jul 07 20:09:00.478047 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.478014 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:00.519799 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.519767 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-config\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.519932 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.519814 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-trusted-ca-bundle\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.519932 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.519839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drs9x\" (UniqueName: \"kubernetes.io/projected/3c993e07-cbae-4e83-ac06-a6a62f0058b5-kube-api-access-drs9x\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.519932 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.519919 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-oauth-config\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.520084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.519943 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-oauth-serving-cert\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.520084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.520012 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-service-ca\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.520084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.520042 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-serving-cert\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.583206 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.583169 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:00.621318 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.621288 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-trusted-ca-bundle\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.621318 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.621321 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drs9x\" (UniqueName: \"kubernetes.io/projected/3c993e07-cbae-4e83-ac06-a6a62f0058b5-kube-api-access-drs9x\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.621522 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.621477 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-oauth-config\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.621586 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.621522 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-oauth-serving-cert\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.621636 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.621591 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-service-ca\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.621689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.621631 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-serving-cert\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.621739 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.621688 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-config\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.622289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.622216 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-oauth-serving-cert\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.622289 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.622224 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-trusted-ca-bundle\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.622438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.622319 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-service-ca\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.622438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.622429 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-config\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.624063 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.624045 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-oauth-config\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.624126 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.624070 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c993e07-cbae-4e83-ac06-a6a62f0058b5-console-serving-cert\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.630181 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.630162 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drs9x\" (UniqueName: \"kubernetes.io/projected/3c993e07-cbae-4e83-ac06-a6a62f0058b5-kube-api-access-drs9x\") pod \"console-bbc4f5cf6-wgc57\" (UID: \"3c993e07-cbae-4e83-ac06-a6a62f0058b5\") " pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.725418 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.725336 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:00.862062 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:00.862034 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbc4f5cf6-wgc57"] Jul 07 20:09:00.863647 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:09:00.863623 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c993e07_cbae_4e83_ac06_a6a62f0058b5.slice/crio-ac74a3e37db6321fb02297efbba7218890e6376c76aead716fbb81520d89173b WatchSource:0}: Error finding container ac74a3e37db6321fb02297efbba7218890e6376c76aead716fbb81520d89173b: Status 404 returned error can't find the container with id ac74a3e37db6321fb02297efbba7218890e6376c76aead716fbb81520d89173b Jul 07 20:09:01.299286 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:01.299248 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbc4f5cf6-wgc57" event={"ID":"3c993e07-cbae-4e83-ac06-a6a62f0058b5","Type":"ContainerStarted","Data":"d13acaf3a4efb3a597ff3c6140579cc2fc479a730e05ab679a5e7dd115197e96"} Jul 07 20:09:01.299664 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:01.299306 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbc4f5cf6-wgc57" event={"ID":"3c993e07-cbae-4e83-ac06-a6a62f0058b5","Type":"ContainerStarted","Data":"ac74a3e37db6321fb02297efbba7218890e6376c76aead716fbb81520d89173b"} Jul 07 20:09:01.327093 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:01.327046 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bbc4f5cf6-wgc57" podStartSLOduration=1.327030762 podStartE2EDuration="1.327030762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:09:01.325055166 +0000 UTC m=+493.833727741" watchObservedRunningTime="2026-07-07 20:09:01.327030762 +0000 UTC m=+493.835703337" Jul 07 20:09:03.050205 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:03.050144 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:03.155328 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:03.155292 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:08.427443 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:08.427407 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:08.487614 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:08.487582 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:08.837978 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:08.837938 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:08.896810 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:08.896774 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:09.289773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:09.289743 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qn5fs" Jul 07 20:09:10.725812 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:10.725779 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:10.726217 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:10.725849 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:10.730676 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:10.730655 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:11.352767 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:11.352740 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bbc4f5cf6-wgc57" Jul 07 20:09:11.410194 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:11.410157 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c84db5bb-59prz"] Jul 07 20:09:11.514199 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:11.514160 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:11.570730 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:11.570698 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:11.634755 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:11.634664 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:12.454465 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:12.454435 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:12.648291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:12.648259 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:12.756588 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:12.756547 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:15.805689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:15.805637 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:15.906526 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:15.906493 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:16.046322 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:16.046274 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:16.146779 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:16.146694 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:17.830129 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:17.830097 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:17.947404 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:17.947375 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:22.688058 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:22.688024 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:22.789723 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:22.789683 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:28.115653 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:28.115612 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:28.214566 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:28.214528 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:28.482660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:28.482627 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:28.540362 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:28.540333 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:32.297527 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:32.297487 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:32.400840 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:32.400798 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:35.650568 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:35.650535 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:35.752794 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:35.752763 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:36.439488 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.439450 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-77c84db5bb-59prz" podUID="ed0ec38e-ff71-43c5-8690-1c629bc895eb" containerName="console" containerID="cri-o://12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b" gracePeriod=15 Jul 07 20:09:36.684931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.684908 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:09:36.838170 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838136 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-oauth-config\") pod \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " Jul 07 20:09:36.838170 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838172 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgsdt\" (UniqueName: \"kubernetes.io/projected/ed0ec38e-ff71-43c5-8690-1c629bc895eb-kube-api-access-xgsdt\") pod \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " Jul 07 20:09:36.838395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838226 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-service-ca\") pod \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " Jul 07 20:09:36.838395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838273 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-config\") pod \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " Jul 07 20:09:36.838395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838314 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-trusted-ca-bundle\") pod \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " Jul 07 20:09:36.838395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838357 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-oauth-serving-cert\") pod \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " Jul 07 20:09:36.838621 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838400 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-serving-cert\") pod \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\" (UID: \"ed0ec38e-ff71-43c5-8690-1c629bc895eb\") " Jul 07 20:09:36.838748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838720 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "ed0ec38e-ff71-43c5-8690-1c629bc895eb" (UID: "ed0ec38e-ff71-43c5-8690-1c629bc895eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:09:36.838829 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838728 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-config" (OuterVolumeSpecName: "console-config") pod "ed0ec38e-ff71-43c5-8690-1c629bc895eb" (UID: "ed0ec38e-ff71-43c5-8690-1c629bc895eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:09:36.838829 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838760 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ed0ec38e-ff71-43c5-8690-1c629bc895eb" (UID: "ed0ec38e-ff71-43c5-8690-1c629bc895eb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:09:36.838829 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.838801 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ed0ec38e-ff71-43c5-8690-1c629bc895eb" (UID: "ed0ec38e-ff71-43c5-8690-1c629bc895eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:09:36.840386 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.840366 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ed0ec38e-ff71-43c5-8690-1c629bc895eb" (UID: "ed0ec38e-ff71-43c5-8690-1c629bc895eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:09:36.840461 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.840371 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed0ec38e-ff71-43c5-8690-1c629bc895eb-kube-api-access-xgsdt" (OuterVolumeSpecName: "kube-api-access-xgsdt") pod "ed0ec38e-ff71-43c5-8690-1c629bc895eb" (UID: "ed0ec38e-ff71-43c5-8690-1c629bc895eb"). InnerVolumeSpecName "kube-api-access-xgsdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:09:36.840508 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.840488 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ed0ec38e-ff71-43c5-8690-1c629bc895eb" (UID: "ed0ec38e-ff71-43c5-8690-1c629bc895eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:09:36.939360 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.939326 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-service-ca\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:09:36.939360 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.939355 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:09:36.939360 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.939364 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-trusted-ca-bundle\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:09:36.939578 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.939374 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0ec38e-ff71-43c5-8690-1c629bc895eb-oauth-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:09:36.939578 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.939384 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-serving-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:09:36.939578 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.939392 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0ec38e-ff71-43c5-8690-1c629bc895eb-console-oauth-config\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:09:36.939578 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:36.939400 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgsdt\" (UniqueName: \"kubernetes.io/projected/ed0ec38e-ff71-43c5-8690-1c629bc895eb-kube-api-access-xgsdt\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:09:37.481701 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.481662 2553 generic.go:356] "Generic (PLEG): container finished" podID="ed0ec38e-ff71-43c5-8690-1c629bc895eb" containerID="12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b" exitCode=0 Jul 07 20:09:37.481890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.481728 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c84db5bb-59prz" Jul 07 20:09:37.481890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.481774 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c84db5bb-59prz" event={"ID":"ed0ec38e-ff71-43c5-8690-1c629bc895eb","Type":"ContainerDied","Data":"12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b"} Jul 07 20:09:37.481890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.481830 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c84db5bb-59prz" event={"ID":"ed0ec38e-ff71-43c5-8690-1c629bc895eb","Type":"ContainerDied","Data":"f1bba816a44ed5e33795b2bd03e791dabee95f2ee0feafce5e4569dd6b981537"} Jul 07 20:09:37.481890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.481854 2553 scope.go:117] "RemoveContainer" containerID="12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b" Jul 07 20:09:37.491594 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.491579 2553 scope.go:117] "RemoveContainer" containerID="12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b" Jul 07 20:09:37.491862 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:09:37.491843 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b\": container with ID starting with 12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b not found: ID does not exist" containerID="12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b" Jul 07 20:09:37.491912 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.491869 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b"} err="failed to get container status \"12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b\": rpc error: code = NotFound desc = could not find container \"12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b\": container with ID starting with 12594e991abc34145417af7c227cf74681638f8a4e2182d0a8594b4d5d216e9b not found: ID does not exist" Jul 07 20:09:37.511400 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.511377 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c84db5bb-59prz"] Jul 07 20:09:37.515707 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:37.515688 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77c84db5bb-59prz"] Jul 07 20:09:38.028724 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:38.028689 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed0ec38e-ff71-43c5-8690-1c629bc895eb" path="/var/lib/kubelet/pods/ed0ec38e-ff71-43c5-8690-1c629bc895eb/volumes" Jul 07 20:09:51.991234 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:51.991200 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp"] Jul 07 20:09:51.991701 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:51.991674 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0ec38e-ff71-43c5-8690-1c629bc895eb" containerName="console" Jul 07 20:09:51.991701 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:51.991700 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0ec38e-ff71-43c5-8690-1c629bc895eb" containerName="console" Jul 07 20:09:51.991910 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:51.991823 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed0ec38e-ff71-43c5-8690-1c629bc895eb" containerName="console" Jul 07 20:09:51.995931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:51.995910 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:51.998874 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:51.998853 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gscsb\"" Jul 07 20:09:52.007966 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.007944 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp"] Jul 07 20:09:52.010265 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.010222 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:52.167227 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167197 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167227 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167232 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167258 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfvkr\" (UniqueName: \"kubernetes.io/projected/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-kube-api-access-nfvkr\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167297 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167376 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167430 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167568 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167494 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167568 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167561 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.167641 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.167626 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.191476 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.191444 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:09:52.268493 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268386 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.268493 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268427 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.268493 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268465 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.268812 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268666 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.268812 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268723 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfvkr\" (UniqueName: \"kubernetes.io/projected/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-kube-api-access-nfvkr\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.268931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268825 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.268931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268894 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.269092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268929 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.269092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.268945 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.269092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.269022 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.269290 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.269256 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.269421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.269314 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.269421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.269332 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.269421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.269411 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.270797 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.270775 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.271146 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.271127 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.276867 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.276837 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.276945 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.276934 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfvkr\" (UniqueName: \"kubernetes.io/projected/f885ca44-9e68-4b8e-a9a2-e58e58f76b27-kube-api-access-nfvkr\") pod \"maas-default-gateway-openshift-default-56654dcf7-7zsnp\" (UID: \"f885ca44-9e68-4b8e-a9a2-e58e58f76b27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.307943 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.307917 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:52.445534 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.445509 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp"] Jul 07 20:09:52.447850 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:09:52.447821 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf885ca44_9e68_4b8e_a9a2_e58e58f76b27.slice/crio-1de76e0a56001d1064cde92544238b924dc897382f6010e2d39a5df884a019ef WatchSource:0}: Error finding container 1de76e0a56001d1064cde92544238b924dc897382f6010e2d39a5df884a019ef: Status 404 returned error can't find the container with id 1de76e0a56001d1064cde92544238b924dc897382f6010e2d39a5df884a019ef Jul 07 20:09:52.450127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.450099 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:09:52.450218 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.450153 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:09:52.450218 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.450183 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:09:52.560200 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.560102 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" event={"ID":"f885ca44-9e68-4b8e-a9a2-e58e58f76b27","Type":"ContainerStarted","Data":"bc87e29126acaeda951e85a05396e20784def43f3e3a382baaf4814dde8fe2b7"} Jul 07 20:09:52.560200 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:52.560167 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" event={"ID":"f885ca44-9e68-4b8e-a9a2-e58e58f76b27","Type":"ContainerStarted","Data":"1de76e0a56001d1064cde92544238b924dc897382f6010e2d39a5df884a019ef"} Jul 07 20:09:53.308241 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:53.308201 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:53.312747 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:53.312725 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:53.335800 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:53.335743 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" podStartSLOduration=2.335727889 podStartE2EDuration="2.335727889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:09:52.582326913 +0000 UTC m=+545.090999485" watchObservedRunningTime="2026-07-07 20:09:53.335727889 +0000 UTC m=+545.844400465" Jul 07 20:09:53.565540 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:53.565462 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:09:53.566411 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:09:53.566391 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:10:06.038706 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.038662 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:06.149528 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.149490 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:06.247955 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.247917 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:06.882793 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.882747 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-dv2lj"] Jul 07 20:10:06.887483 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.887457 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dv2lj" Jul 07 20:10:06.890335 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.890310 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9xd54\"" Jul 07 20:10:06.893792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.893767 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dv2lj"] Jul 07 20:10:06.910599 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.910575 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:06.983684 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:06.983650 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:07.001437 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.001408 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sjxd\" (UniqueName: \"kubernetes.io/projected/ea60b331-50b0-4784-ba4b-5aa07a294711-kube-api-access-6sjxd\") pod \"authorino-58cd955675-dv2lj\" (UID: \"ea60b331-50b0-4784-ba4b-5aa07a294711\") " pod="kuadrant-system/authorino-58cd955675-dv2lj" Jul 07 20:10:07.094576 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.094541 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-zndh7"] Jul 07 20:10:07.098212 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.098193 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zndh7" Jul 07 20:10:07.102093 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.102068 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjxd\" (UniqueName: \"kubernetes.io/projected/ea60b331-50b0-4784-ba4b-5aa07a294711-kube-api-access-6sjxd\") pod \"authorino-58cd955675-dv2lj\" (UID: \"ea60b331-50b0-4784-ba4b-5aa07a294711\") " pod="kuadrant-system/authorino-58cd955675-dv2lj" Jul 07 20:10:07.102634 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.102610 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zndh7"] Jul 07 20:10:07.109985 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.109962 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjxd\" (UniqueName: \"kubernetes.io/projected/ea60b331-50b0-4784-ba4b-5aa07a294711-kube-api-access-6sjxd\") pod \"authorino-58cd955675-dv2lj\" (UID: \"ea60b331-50b0-4784-ba4b-5aa07a294711\") " pod="kuadrant-system/authorino-58cd955675-dv2lj" Jul 07 20:10:07.198364 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.198282 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dv2lj" Jul 07 20:10:07.202515 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.202495 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99r5n\" (UniqueName: \"kubernetes.io/projected/b70a2bd0-c9c5-4bad-aba0-bd0b64967d34-kube-api-access-99r5n\") pod \"authorino-75d556bd77-zndh7\" (UID: \"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34\") " pod="kuadrant-system/authorino-75d556bd77-zndh7" Jul 07 20:10:07.303785 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.303749 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99r5n\" (UniqueName: \"kubernetes.io/projected/b70a2bd0-c9c5-4bad-aba0-bd0b64967d34-kube-api-access-99r5n\") pod \"authorino-75d556bd77-zndh7\" (UID: \"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34\") " pod="kuadrant-system/authorino-75d556bd77-zndh7" Jul 07 20:10:07.312601 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.312564 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99r5n\" (UniqueName: \"kubernetes.io/projected/b70a2bd0-c9c5-4bad-aba0-bd0b64967d34-kube-api-access-99r5n\") pod \"authorino-75d556bd77-zndh7\" (UID: \"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34\") " pod="kuadrant-system/authorino-75d556bd77-zndh7" Jul 07 20:10:07.332518 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.332473 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dv2lj"] Jul 07 20:10:07.334158 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:10:07.334131 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea60b331_50b0_4784_ba4b_5aa07a294711.slice/crio-8bdcea806b40fae9209fed969e29f9b6fd62fb9f1bea790a99df5dd45bff684b WatchSource:0}: Error finding container 8bdcea806b40fae9209fed969e29f9b6fd62fb9f1bea790a99df5dd45bff684b: Status 404 returned error can't find the container with id 8bdcea806b40fae9209fed969e29f9b6fd62fb9f1bea790a99df5dd45bff684b Jul 07 20:10:07.407394 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.407358 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zndh7" Jul 07 20:10:07.539441 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.539416 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zndh7"] Jul 07 20:10:07.541398 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:10:07.541370 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb70a2bd0_c9c5_4bad_aba0_bd0b64967d34.slice/crio-fab890af6d4578ba7d09b1d5ff5590c7eb3cfad6c14a8d6aefeb734bd273e033 WatchSource:0}: Error finding container fab890af6d4578ba7d09b1d5ff5590c7eb3cfad6c14a8d6aefeb734bd273e033: Status 404 returned error can't find the container with id fab890af6d4578ba7d09b1d5ff5590c7eb3cfad6c14a8d6aefeb734bd273e033 Jul 07 20:10:07.636581 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.636542 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zndh7" event={"ID":"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34","Type":"ContainerStarted","Data":"fab890af6d4578ba7d09b1d5ff5590c7eb3cfad6c14a8d6aefeb734bd273e033"} Jul 07 20:10:07.637616 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:07.637588 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dv2lj" event={"ID":"ea60b331-50b0-4784-ba4b-5aa07a294711","Type":"ContainerStarted","Data":"8bdcea806b40fae9209fed969e29f9b6fd62fb9f1bea790a99df5dd45bff684b"} Jul 07 20:10:11.666867 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:11.666823 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dv2lj" event={"ID":"ea60b331-50b0-4784-ba4b-5aa07a294711","Type":"ContainerStarted","Data":"cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed"} Jul 07 20:10:11.668258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:11.668230 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zndh7" event={"ID":"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34","Type":"ContainerStarted","Data":"86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811"} Jul 07 20:10:11.684379 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:11.684333 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-dv2lj" podStartSLOduration=1.899484597 podStartE2EDuration="5.684319237s" totalImagesPullingTime="3.78483464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:10:11.683009699 +0000 UTC m=+564.191682272" watchObservedRunningTime="2026-07-07 20:10:11.684319237 +0000 UTC m=+564.192991850" Jul 07 20:10:11.699226 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:11.699181 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-zndh7" podStartSLOduration=1.132793303 podStartE2EDuration="4.699165859s" totalImagesPullingTime="3.566372556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:10:11.697685412 +0000 UTC m=+564.206357989" watchObservedRunningTime="2026-07-07 20:10:11.699165859 +0000 UTC m=+564.207838433" Jul 07 20:10:11.726882 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:11.726849 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dv2lj"] Jul 07 20:10:13.679065 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:13.679025 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-dv2lj" podUID="ea60b331-50b0-4784-ba4b-5aa07a294711" containerName="authorino" containerID="cri-o://cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed" gracePeriod=30 Jul 07 20:10:13.928039 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:13.928016 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dv2lj" Jul 07 20:10:13.965869 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:13.965807 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sjxd\" (UniqueName: \"kubernetes.io/projected/ea60b331-50b0-4784-ba4b-5aa07a294711-kube-api-access-6sjxd\") pod \"ea60b331-50b0-4784-ba4b-5aa07a294711\" (UID: \"ea60b331-50b0-4784-ba4b-5aa07a294711\") " Jul 07 20:10:13.967711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:13.967687 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea60b331-50b0-4784-ba4b-5aa07a294711-kube-api-access-6sjxd" (OuterVolumeSpecName: "kube-api-access-6sjxd") pod "ea60b331-50b0-4784-ba4b-5aa07a294711" (UID: "ea60b331-50b0-4784-ba4b-5aa07a294711"). InnerVolumeSpecName "kube-api-access-6sjxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:10:14.066769 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.066742 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sjxd\" (UniqueName: \"kubernetes.io/projected/ea60b331-50b0-4784-ba4b-5aa07a294711-kube-api-access-6sjxd\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:10:14.685336 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.685300 2553 generic.go:356] "Generic (PLEG): container finished" podID="ea60b331-50b0-4784-ba4b-5aa07a294711" containerID="cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed" exitCode=0 Jul 07 20:10:14.685773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.685367 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dv2lj" Jul 07 20:10:14.685773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.685389 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dv2lj" event={"ID":"ea60b331-50b0-4784-ba4b-5aa07a294711","Type":"ContainerDied","Data":"cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed"} Jul 07 20:10:14.685773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.685441 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dv2lj" event={"ID":"ea60b331-50b0-4784-ba4b-5aa07a294711","Type":"ContainerDied","Data":"8bdcea806b40fae9209fed969e29f9b6fd62fb9f1bea790a99df5dd45bff684b"} Jul 07 20:10:14.685773 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.685465 2553 scope.go:117] "RemoveContainer" containerID="cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed" Jul 07 20:10:14.693971 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.693955 2553 scope.go:117] "RemoveContainer" containerID="cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed" Jul 07 20:10:14.694217 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:10:14.694196 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed\": container with ID starting with cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed not found: ID does not exist" containerID="cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed" Jul 07 20:10:14.694318 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.694222 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed"} err="failed to get container status \"cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed\": rpc error: code = NotFound desc = could not find container \"cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed\": container with ID starting with cc68bb98b1da5308480b504c437355bdbda3358f8788f5b31016a70c4845e9ed not found: ID does not exist" Jul 07 20:10:14.702689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.702667 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dv2lj"] Jul 07 20:10:14.709784 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:14.709764 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-dv2lj"] Jul 07 20:10:16.029537 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:16.029504 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea60b331-50b0-4784-ba4b-5aa07a294711" path="/var/lib/kubelet/pods/ea60b331-50b0-4784-ba4b-5aa07a294711/volumes" Jul 07 20:10:26.023689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:26.023633 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:38.132930 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.132893 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:38.292863 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.292827 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:38.380190 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.380152 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dfqpz"] Jul 07 20:10:38.380540 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.380526 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea60b331-50b0-4784-ba4b-5aa07a294711" containerName="authorino" Jul 07 20:10:38.380595 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.380542 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea60b331-50b0-4784-ba4b-5aa07a294711" containerName="authorino" Jul 07 20:10:38.380638 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.380628 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea60b331-50b0-4784-ba4b-5aa07a294711" containerName="authorino" Jul 07 20:10:38.386362 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.386307 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dfqpz" Jul 07 20:10:38.395838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.395817 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dfqpz"] Jul 07 20:10:38.581348 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.581300 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56jd2\" (UniqueName: \"kubernetes.io/projected/2a7d09b1-9ad6-4d0c-9088-7f04983094f1-kube-api-access-56jd2\") pod \"authorino-5d4b67c446-dfqpz\" (UID: \"2a7d09b1-9ad6-4d0c-9088-7f04983094f1\") " pod="kuadrant-system/authorino-5d4b67c446-dfqpz" Jul 07 20:10:38.632882 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.632843 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dfqpz"] Jul 07 20:10:38.633192 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:10:38.633166 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-56jd2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-dfqpz" podUID="2a7d09b1-9ad6-4d0c-9088-7f04983094f1" Jul 07 20:10:38.660488 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.660400 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84b7dcdfd9-ldbp9"] Jul 07 20:10:38.663780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.663763 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" Jul 07 20:10:38.671806 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.671785 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84b7dcdfd9-ldbp9"] Jul 07 20:10:38.682030 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.682008 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w87hv\" (UniqueName: \"kubernetes.io/projected/b7050b26-05e8-4022-96f3-ebec7c19e0e8-kube-api-access-w87hv\") pod \"authorino-84b7dcdfd9-ldbp9\" (UID: \"b7050b26-05e8-4022-96f3-ebec7c19e0e8\") " pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" Jul 07 20:10:38.682130 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.682053 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56jd2\" (UniqueName: \"kubernetes.io/projected/2a7d09b1-9ad6-4d0c-9088-7f04983094f1-kube-api-access-56jd2\") pod \"authorino-5d4b67c446-dfqpz\" (UID: \"2a7d09b1-9ad6-4d0c-9088-7f04983094f1\") " pod="kuadrant-system/authorino-5d4b67c446-dfqpz" Jul 07 20:10:38.690775 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.690751 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56jd2\" (UniqueName: \"kubernetes.io/projected/2a7d09b1-9ad6-4d0c-9088-7f04983094f1-kube-api-access-56jd2\") pod \"authorino-5d4b67c446-dfqpz\" (UID: \"2a7d09b1-9ad6-4d0c-9088-7f04983094f1\") " pod="kuadrant-system/authorino-5d4b67c446-dfqpz" Jul 07 20:10:38.740216 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.740178 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84b7dcdfd9-ldbp9"] Jul 07 20:10:38.740478 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:10:38.740459 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-w87hv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" podUID="b7050b26-05e8-4022-96f3-ebec7c19e0e8" Jul 07 20:10:38.770969 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.770926 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79f897cdbf-dhf2m"] Jul 07 20:10:38.774303 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.774287 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:38.776966 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.776950 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 20:10:38.782307 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.782274 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79f897cdbf-dhf2m"] Jul 07 20:10:38.782664 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.782635 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w87hv\" (UniqueName: \"kubernetes.io/projected/b7050b26-05e8-4022-96f3-ebec7c19e0e8-kube-api-access-w87hv\") pod \"authorino-84b7dcdfd9-ldbp9\" (UID: \"b7050b26-05e8-4022-96f3-ebec7c19e0e8\") " pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" Jul 07 20:10:38.782776 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.782726 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00a6dc33-e909-44b3-b302-5af5067127d8-tls-cert\") pod \"authorino-79f897cdbf-dhf2m\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:38.782844 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.782787 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrvsn\" (UniqueName: \"kubernetes.io/projected/00a6dc33-e909-44b3-b302-5af5067127d8-kube-api-access-wrvsn\") pod \"authorino-79f897cdbf-dhf2m\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:38.794386 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.794357 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w87hv\" (UniqueName: \"kubernetes.io/projected/b7050b26-05e8-4022-96f3-ebec7c19e0e8-kube-api-access-w87hv\") pod \"authorino-84b7dcdfd9-ldbp9\" (UID: \"b7050b26-05e8-4022-96f3-ebec7c19e0e8\") " pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" Jul 07 20:10:38.807293 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.807270 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" Jul 07 20:10:38.807421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.807274 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dfqpz" Jul 07 20:10:38.814061 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.814043 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" Jul 07 20:10:38.816861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.816846 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dfqpz" Jul 07 20:10:38.883490 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.883449 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00a6dc33-e909-44b3-b302-5af5067127d8-tls-cert\") pod \"authorino-79f897cdbf-dhf2m\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:38.883702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.883507 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrvsn\" (UniqueName: \"kubernetes.io/projected/00a6dc33-e909-44b3-b302-5af5067127d8-kube-api-access-wrvsn\") pod \"authorino-79f897cdbf-dhf2m\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:38.885980 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.885957 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00a6dc33-e909-44b3-b302-5af5067127d8-tls-cert\") pod \"authorino-79f897cdbf-dhf2m\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:38.893190 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.893164 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrvsn\" (UniqueName: \"kubernetes.io/projected/00a6dc33-e909-44b3-b302-5af5067127d8-kube-api-access-wrvsn\") pod \"authorino-79f897cdbf-dhf2m\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:38.984623 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.984583 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w87hv\" (UniqueName: \"kubernetes.io/projected/b7050b26-05e8-4022-96f3-ebec7c19e0e8-kube-api-access-w87hv\") pod \"b7050b26-05e8-4022-96f3-ebec7c19e0e8\" (UID: \"b7050b26-05e8-4022-96f3-ebec7c19e0e8\") " Jul 07 20:10:38.984809 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.984709 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56jd2\" (UniqueName: \"kubernetes.io/projected/2a7d09b1-9ad6-4d0c-9088-7f04983094f1-kube-api-access-56jd2\") pod \"2a7d09b1-9ad6-4d0c-9088-7f04983094f1\" (UID: \"2a7d09b1-9ad6-4d0c-9088-7f04983094f1\") " Jul 07 20:10:38.986694 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.986663 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7050b26-05e8-4022-96f3-ebec7c19e0e8-kube-api-access-w87hv" (OuterVolumeSpecName: "kube-api-access-w87hv") pod "b7050b26-05e8-4022-96f3-ebec7c19e0e8" (UID: "b7050b26-05e8-4022-96f3-ebec7c19e0e8"). InnerVolumeSpecName "kube-api-access-w87hv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:10:38.986694 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:38.986674 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a7d09b1-9ad6-4d0c-9088-7f04983094f1-kube-api-access-56jd2" (OuterVolumeSpecName: "kube-api-access-56jd2") pod "2a7d09b1-9ad6-4d0c-9088-7f04983094f1" (UID: "2a7d09b1-9ad6-4d0c-9088-7f04983094f1"). InnerVolumeSpecName "kube-api-access-56jd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:10:39.085021 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.084976 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:10:39.085922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.085885 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w87hv\" (UniqueName: \"kubernetes.io/projected/b7050b26-05e8-4022-96f3-ebec7c19e0e8-kube-api-access-w87hv\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:10:39.085922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.085912 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56jd2\" (UniqueName: \"kubernetes.io/projected/2a7d09b1-9ad6-4d0c-9088-7f04983094f1-kube-api-access-56jd2\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:10:39.218457 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.218430 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79f897cdbf-dhf2m"] Jul 07 20:10:39.219638 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:10:39.219613 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00a6dc33_e909_44b3_b302_5af5067127d8.slice/crio-928a8d0acf72e3e2c29ee643f2cf70c6fcf109a8becb0e196d1c12df8f25c491 WatchSource:0}: Error finding container 928a8d0acf72e3e2c29ee643f2cf70c6fcf109a8becb0e196d1c12df8f25c491: Status 404 returned error can't find the container with id 928a8d0acf72e3e2c29ee643f2cf70c6fcf109a8becb0e196d1c12df8f25c491 Jul 07 20:10:39.814873 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.814824 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" event={"ID":"00a6dc33-e909-44b3-b302-5af5067127d8","Type":"ContainerStarted","Data":"a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d"} Jul 07 20:10:39.815030 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.814884 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" event={"ID":"00a6dc33-e909-44b3-b302-5af5067127d8","Type":"ContainerStarted","Data":"928a8d0acf72e3e2c29ee643f2cf70c6fcf109a8becb0e196d1c12df8f25c491"} Jul 07 20:10:39.815114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.815042 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dfqpz" Jul 07 20:10:39.815176 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.815154 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b7dcdfd9-ldbp9" Jul 07 20:10:39.832731 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.832684 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" podStartSLOduration=1.316294191 podStartE2EDuration="1.832669296s" totalImagesPullingTime="516.375105ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:10:39.831036389 +0000 UTC m=+592.339708967" watchObservedRunningTime="2026-07-07 20:10:39.832669296 +0000 UTC m=+592.341341871" Jul 07 20:10:39.856178 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.856135 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dfqpz"] Jul 07 20:10:39.863192 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.863166 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dfqpz"] Jul 07 20:10:39.865733 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.865710 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zndh7"] Jul 07 20:10:39.865927 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.865904 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-zndh7" podUID="b70a2bd0-c9c5-4bad-aba0-bd0b64967d34" containerName="authorino" containerID="cri-o://86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811" gracePeriod=30 Jul 07 20:10:39.894202 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.894171 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84b7dcdfd9-ldbp9"] Jul 07 20:10:39.897319 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:39.897280 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84b7dcdfd9-ldbp9"] Jul 07 20:10:40.031071 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.030842 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a7d09b1-9ad6-4d0c-9088-7f04983094f1" path="/var/lib/kubelet/pods/2a7d09b1-9ad6-4d0c-9088-7f04983094f1/volumes" Jul 07 20:10:40.031232 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.031210 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7050b26-05e8-4022-96f3-ebec7c19e0e8" path="/var/lib/kubelet/pods/b7050b26-05e8-4022-96f3-ebec7c19e0e8/volumes" Jul 07 20:10:40.099498 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.099477 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zndh7" Jul 07 20:10:40.195302 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.195268 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99r5n\" (UniqueName: \"kubernetes.io/projected/b70a2bd0-c9c5-4bad-aba0-bd0b64967d34-kube-api-access-99r5n\") pod \"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34\" (UID: \"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34\") " Jul 07 20:10:40.197287 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.197264 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70a2bd0-c9c5-4bad-aba0-bd0b64967d34-kube-api-access-99r5n" (OuterVolumeSpecName: "kube-api-access-99r5n") pod "b70a2bd0-c9c5-4bad-aba0-bd0b64967d34" (UID: "b70a2bd0-c9c5-4bad-aba0-bd0b64967d34"). InnerVolumeSpecName "kube-api-access-99r5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:10:40.295836 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.295768 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99r5n\" (UniqueName: \"kubernetes.io/projected/b70a2bd0-c9c5-4bad-aba0-bd0b64967d34-kube-api-access-99r5n\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:10:40.821142 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.821112 2553 generic.go:356] "Generic (PLEG): container finished" podID="b70a2bd0-c9c5-4bad-aba0-bd0b64967d34" containerID="86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811" exitCode=0 Jul 07 20:10:40.821306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.821154 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zndh7" Jul 07 20:10:40.821306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.821198 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zndh7" event={"ID":"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34","Type":"ContainerDied","Data":"86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811"} Jul 07 20:10:40.821306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.821235 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zndh7" event={"ID":"b70a2bd0-c9c5-4bad-aba0-bd0b64967d34","Type":"ContainerDied","Data":"fab890af6d4578ba7d09b1d5ff5590c7eb3cfad6c14a8d6aefeb734bd273e033"} Jul 07 20:10:40.821306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.821251 2553 scope.go:117] "RemoveContainer" containerID="86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811" Jul 07 20:10:40.829883 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.829867 2553 scope.go:117] "RemoveContainer" containerID="86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811" Jul 07 20:10:40.830151 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:10:40.830132 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811\": container with ID starting with 86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811 not found: ID does not exist" containerID="86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811" Jul 07 20:10:40.830232 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.830161 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811"} err="failed to get container status \"86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811\": rpc error: code = NotFound desc = could not find container \"86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811\": container with ID starting with 86121e202514460e5f628b08e6ad434f01aa6a147b18594305327c4d1fa6d811 not found: ID does not exist" Jul 07 20:10:40.843297 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.843276 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zndh7"] Jul 07 20:10:40.849034 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:40.849015 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zndh7"] Jul 07 20:10:42.028734 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:42.028701 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70a2bd0-c9c5-4bad-aba0-bd0b64967d34" path="/var/lib/kubelet/pods/b70a2bd0-c9c5-4bad-aba0-bd0b64967d34/volumes" Jul 07 20:10:44.448605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.448559 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:44.461048 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.461018 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f9ff6bd79-59vt4"] Jul 07 20:10:44.461359 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.461347 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70a2bd0-c9c5-4bad-aba0-bd0b64967d34" containerName="authorino" Jul 07 20:10:44.461400 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.461360 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70a2bd0-c9c5-4bad-aba0-bd0b64967d34" containerName="authorino" Jul 07 20:10:44.461441 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.461432 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b70a2bd0-c9c5-4bad-aba0-bd0b64967d34" containerName="authorino" Jul 07 20:10:44.464369 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.464348 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.468199 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.468177 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qdcdp\"" Jul 07 20:10:44.468970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.468950 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 20:10:44.469870 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.469855 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 20:10:44.476219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.476199 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f9ff6bd79-59vt4"] Jul 07 20:10:44.530399 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.530366 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd821d9-3ddf-4d4a-937e-f678e6a54809-webhook-cert\") pod \"maas-controller-6f9ff6bd79-59vt4\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.530596 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.530430 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8klj\" (UniqueName: \"kubernetes.io/projected/6bd821d9-3ddf-4d4a-937e-f678e6a54809-kube-api-access-t8klj\") pod \"maas-controller-6f9ff6bd79-59vt4\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.548194 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.548158 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:44.631061 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.631028 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8klj\" (UniqueName: \"kubernetes.io/projected/6bd821d9-3ddf-4d4a-937e-f678e6a54809-kube-api-access-t8klj\") pod \"maas-controller-6f9ff6bd79-59vt4\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.631213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.631109 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd821d9-3ddf-4d4a-937e-f678e6a54809-webhook-cert\") pod \"maas-controller-6f9ff6bd79-59vt4\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.633679 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.633658 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd821d9-3ddf-4d4a-937e-f678e6a54809-webhook-cert\") pod \"maas-controller-6f9ff6bd79-59vt4\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.642070 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.642045 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8klj\" (UniqueName: \"kubernetes.io/projected/6bd821d9-3ddf-4d4a-937e-f678e6a54809-kube-api-access-t8klj\") pod \"maas-controller-6f9ff6bd79-59vt4\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.645478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.645453 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:44.777549 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.777514 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:44.919311 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:44.919284 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f9ff6bd79-59vt4"] Jul 07 20:10:44.920660 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:10:44.920626 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd821d9_3ddf_4d4a_937e_f678e6a54809.slice/crio-a9a2d8faba67d0d0500fe13854ede152b5bb430c205a754ccfde843dd0a0470b WatchSource:0}: Error finding container a9a2d8faba67d0d0500fe13854ede152b5bb430c205a754ccfde843dd0a0470b: Status 404 returned error can't find the container with id a9a2d8faba67d0d0500fe13854ede152b5bb430c205a754ccfde843dd0a0470b Jul 07 20:10:45.854651 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:45.854605 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" event={"ID":"6bd821d9-3ddf-4d4a-937e-f678e6a54809","Type":"ContainerStarted","Data":"a9a2d8faba67d0d0500fe13854ede152b5bb430c205a754ccfde843dd0a0470b"} Jul 07 20:10:47.719476 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:47.719436 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:47.838620 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:47.838531 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:10:47.866428 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:47.866399 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" event={"ID":"6bd821d9-3ddf-4d4a-937e-f678e6a54809","Type":"ContainerStarted","Data":"37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b"} Jul 07 20:10:47.866657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:47.866457 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:47.886583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:47.886533 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" podStartSLOduration=1.2977762880000001 podStartE2EDuration="3.886518124s" totalImagesPullingTime="2.588741836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:10:47.885775325 +0000 UTC m=+600.394447899" watchObservedRunningTime="2026-07-07 20:10:47.886518124 +0000 UTC m=+600.395190758" Jul 07 20:10:47.953764 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:47.953738 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:10:47.957566 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:47.957539 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:10:58.878213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:58.878180 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:58.890421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:58.890336 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f9ff6bd79-59vt4"] Jul 07 20:10:58.924497 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:58.924458 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" podUID="6bd821d9-3ddf-4d4a-937e-f678e6a54809" containerName="manager" containerID="cri-o://37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b" gracePeriod=30 Jul 07 20:10:59.174671 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.174648 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:59.255454 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.255407 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8klj\" (UniqueName: \"kubernetes.io/projected/6bd821d9-3ddf-4d4a-937e-f678e6a54809-kube-api-access-t8klj\") pod \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " Jul 07 20:10:59.255653 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.255472 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd821d9-3ddf-4d4a-937e-f678e6a54809-webhook-cert\") pod \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\" (UID: \"6bd821d9-3ddf-4d4a-937e-f678e6a54809\") " Jul 07 20:10:59.257551 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.257523 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bd821d9-3ddf-4d4a-937e-f678e6a54809-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6bd821d9-3ddf-4d4a-937e-f678e6a54809" (UID: "6bd821d9-3ddf-4d4a-937e-f678e6a54809"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:10:59.257660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.257586 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd821d9-3ddf-4d4a-937e-f678e6a54809-kube-api-access-t8klj" (OuterVolumeSpecName: "kube-api-access-t8klj") pod "6bd821d9-3ddf-4d4a-937e-f678e6a54809" (UID: "6bd821d9-3ddf-4d4a-937e-f678e6a54809"). InnerVolumeSpecName "kube-api-access-t8klj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:10:59.356122 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.356080 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8klj\" (UniqueName: \"kubernetes.io/projected/6bd821d9-3ddf-4d4a-937e-f678e6a54809-kube-api-access-t8klj\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:10:59.356122 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.356118 2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bd821d9-3ddf-4d4a-937e-f678e6a54809-webhook-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:10:59.931160 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.931124 2553 generic.go:356] "Generic (PLEG): container finished" podID="6bd821d9-3ddf-4d4a-937e-f678e6a54809" containerID="37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b" exitCode=0 Jul 07 20:10:59.931548 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.931167 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" Jul 07 20:10:59.931548 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.931164 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" event={"ID":"6bd821d9-3ddf-4d4a-937e-f678e6a54809","Type":"ContainerDied","Data":"37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b"} Jul 07 20:10:59.931548 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.931291 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9ff6bd79-59vt4" event={"ID":"6bd821d9-3ddf-4d4a-937e-f678e6a54809","Type":"ContainerDied","Data":"a9a2d8faba67d0d0500fe13854ede152b5bb430c205a754ccfde843dd0a0470b"} Jul 07 20:10:59.931548 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.931324 2553 scope.go:117] "RemoveContainer" containerID="37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b" Jul 07 20:10:59.940464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.940448 2553 scope.go:117] "RemoveContainer" containerID="37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b" Jul 07 20:10:59.940737 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:10:59.940719 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b\": container with ID starting with 37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b not found: ID does not exist" containerID="37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b" Jul 07 20:10:59.940799 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.940744 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b"} err="failed to get container status \"37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b\": rpc error: code = NotFound desc = could not find container \"37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b\": container with ID starting with 37d3e8789ba5d71ad632381f426bc04d4b234cf6e36b49ef7126c1f1df58e79b not found: ID does not exist" Jul 07 20:10:59.954637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.954616 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f9ff6bd79-59vt4"] Jul 07 20:10:59.960378 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:10:59.960353 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f9ff6bd79-59vt4"] Jul 07 20:11:00.029260 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:00.029226 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd821d9-3ddf-4d4a-937e-f678e6a54809" path="/var/lib/kubelet/pods/6bd821d9-3ddf-4d4a-937e-f678e6a54809/volumes" Jul 07 20:11:18.064964 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:18.064926 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:19.418655 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:19.418613 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:19.660917 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:19.660884 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:20.107897 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.107863 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:20.207381 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.207343 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:20.224110 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.224085 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jxth8"] Jul 07 20:11:20.224442 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.224428 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd821d9-3ddf-4d4a-937e-f678e6a54809" containerName="manager" Jul 07 20:11:20.224486 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.224444 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd821d9-3ddf-4d4a-937e-f678e6a54809" containerName="manager" Jul 07 20:11:20.224524 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.224518 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd821d9-3ddf-4d4a-937e-f678e6a54809" containerName="manager" Jul 07 20:11:20.229826 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.229795 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.232409 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.232388 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 20:11:20.232557 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.232438 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-d7h2x\"" Jul 07 20:11:20.237965 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.237942 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jxth8"] Jul 07 20:11:20.346342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.346308 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85lj5\" (UniqueName: \"kubernetes.io/projected/6a828525-b750-4c87-a09b-4b8052369d30-kube-api-access-85lj5\") pod \"payload-pre-processing-7964457696-jxth8\" (UID: \"6a828525-b750-4c87-a09b-4b8052369d30\") " pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.346492 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.346355 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a828525-b750-4c87-a09b-4b8052369d30-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jxth8\" (UID: \"6a828525-b750-4c87-a09b-4b8052369d30\") " pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.446847 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.446766 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85lj5\" (UniqueName: \"kubernetes.io/projected/6a828525-b750-4c87-a09b-4b8052369d30-kube-api-access-85lj5\") pod \"payload-pre-processing-7964457696-jxth8\" (UID: \"6a828525-b750-4c87-a09b-4b8052369d30\") " pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.446847 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.446814 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a828525-b750-4c87-a09b-4b8052369d30-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jxth8\" (UID: \"6a828525-b750-4c87-a09b-4b8052369d30\") " pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.447615 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.447590 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a828525-b750-4c87-a09b-4b8052369d30-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jxth8\" (UID: \"6a828525-b750-4c87-a09b-4b8052369d30\") " pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.456015 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.455977 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85lj5\" (UniqueName: \"kubernetes.io/projected/6a828525-b750-4c87-a09b-4b8052369d30-kube-api-access-85lj5\") pod \"payload-pre-processing-7964457696-jxth8\" (UID: \"6a828525-b750-4c87-a09b-4b8052369d30\") " pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.540269 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.540236 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:20.671499 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.671476 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jxth8"] Jul 07 20:11:20.673142 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:11:20.673110 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a828525_b750_4c87_a09b_4b8052369d30.slice/crio-afce651ce3afc93e463bf007115d6c109d3f4e8c331406e4ed2713f9969d42f7 WatchSource:0}: Error finding container afce651ce3afc93e463bf007115d6c109d3f4e8c331406e4ed2713f9969d42f7: Status 404 returned error can't find the container with id afce651ce3afc93e463bf007115d6c109d3f4e8c331406e4ed2713f9969d42f7 Jul 07 20:11:20.675041 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:20.675020 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 20:11:21.045175 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:21.045141 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" event={"ID":"6a828525-b750-4c87-a09b-4b8052369d30","Type":"ContainerStarted","Data":"afce651ce3afc93e463bf007115d6c109d3f4e8c331406e4ed2713f9969d42f7"} Jul 07 20:11:23.058958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:23.058910 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" event={"ID":"6a828525-b750-4c87-a09b-4b8052369d30","Type":"ContainerStarted","Data":"5eda2f87a5f82f3e110bf82a88474ccb45681f4ebc522a135708d22f57146b75"} Jul 07 20:11:23.059379 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:23.058985 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:23.077587 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:23.077540 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" podStartSLOduration=1.4160356379999999 podStartE2EDuration="3.07752547s" totalImagesPullingTime="1.661489832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:11:23.075050602 +0000 UTC m=+635.583723177" watchObservedRunningTime="2026-07-07 20:11:23.07752547 +0000 UTC m=+635.586198134" Jul 07 20:11:28.087276 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:28.087200 2553 generic.go:356] "Generic (PLEG): container finished" podID="f885ca44-9e68-4b8e-a9a2-e58e58f76b27" containerID="bc87e29126acaeda951e85a05396e20784def43f3e3a382baaf4814dde8fe2b7" exitCode=0 Jul 07 20:11:28.087276 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:28.087246 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" event={"ID":"f885ca44-9e68-4b8e-a9a2-e58e58f76b27","Type":"ContainerDied","Data":"bc87e29126acaeda951e85a05396e20784def43f3e3a382baaf4814dde8fe2b7"} Jul 07 20:11:28.087728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:28.087602 2553 scope.go:117] "RemoveContainer" containerID="bc87e29126acaeda951e85a05396e20784def43f3e3a382baaf4814dde8fe2b7" Jul 07 20:11:28.088544 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:28.088516 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:11:28.088672 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:28.088604 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:11:28.088736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:28.088669 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:11:29.095048 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:29.095009 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" event={"ID":"f885ca44-9e68-4b8e-a9a2-e58e58f76b27","Type":"ContainerStarted","Data":"d5327345c203691e3fa7d184f9e1627238da3248dd5c72926d8446819366807b"} Jul 07 20:11:29.308717 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:29.308597 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:11:29.308717 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:29.308639 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7zsnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 20:11:29.309104 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:29.308759 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" podUID="f885ca44-9e68-4b8e-a9a2-e58e58f76b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 20:11:30.308721 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:30.308680 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7zsnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 20:11:30.309441 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:30.309410 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" podUID="f885ca44-9e68-4b8e-a9a2-e58e58f76b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 20:11:30.501977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:30.501940 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:30.601630 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:30.601544 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:31.317442 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:31.317396 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7zsnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 20:11:31.317939 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:31.317474 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" podUID="f885ca44-9e68-4b8e-a9a2-e58e58f76b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 20:11:32.308596 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:32.308547 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7zsnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 07 20:11:32.308905 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:32.308873 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" podUID="f885ca44-9e68-4b8e-a9a2-e58e58f76b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 07 20:11:33.312234 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:33.312203 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:11:33.312595 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:33.312576 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:11:33.313234 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:33.313215 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7zsnp" Jul 07 20:11:34.065093 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:34.065051 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" podUID="6a828525-b750-4c87-a09b-4b8052369d30" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.52:9004: connect: connection refused" Jul 07 20:11:42.162010 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:42.161946 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:44.064482 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:44.064447 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-jxth8" Jul 07 20:11:49.467212 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.467177 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn"] Jul 07 20:11:49.471786 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.471764 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.475880 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.475856 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 20:11:49.476031 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.475888 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8k5vx\"" Jul 07 20:11:49.476031 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.475896 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 20:11:49.476031 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.475944 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 20:11:49.484165 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.484128 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn"] Jul 07 20:11:49.494744 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.494714 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:11:49.617223 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.617192 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.617223 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.617232 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.617452 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.617256 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkbjg\" (UniqueName: \"kubernetes.io/projected/9b6b01a9-25ce-4982-ad5d-d005c928122c-kube-api-access-wkbjg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.617452 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.617359 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.617452 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.617379 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.617452 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.617393 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6b01a9-25ce-4982-ad5d-d005c928122c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.718672 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.718579 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkbjg\" (UniqueName: \"kubernetes.io/projected/9b6b01a9-25ce-4982-ad5d-d005c928122c-kube-api-access-wkbjg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.718846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.718683 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.718846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.718717 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.718846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.718738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6b01a9-25ce-4982-ad5d-d005c928122c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.718846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.718781 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.718846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.718813 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.719169 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.719139 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.719304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.719230 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.719304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.719277 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.721105 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.721083 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6b01a9-25ce-4982-ad5d-d005c928122c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.721356 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.721339 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6b01a9-25ce-4982-ad5d-d005c928122c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.727060 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.727040 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkbjg\" (UniqueName: \"kubernetes.io/projected/9b6b01a9-25ce-4982-ad5d-d005c928122c-kube-api-access-wkbjg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-77ljn\" (UID: \"9b6b01a9-25ce-4982-ad5d-d005c928122c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.790066 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.790026 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:49.933390 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:49.933365 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn"] Jul 07 20:11:49.934738 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:11:49.934713 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6b01a9_25ce_4982_ad5d_d005c928122c.slice/crio-6786812a3e942caa568831a5359459ebb63ccdb6b1ae1c9fbd4ecfeac4abbcdd WatchSource:0}: Error finding container 6786812a3e942caa568831a5359459ebb63ccdb6b1ae1c9fbd4ecfeac4abbcdd: Status 404 returned error can't find the container with id 6786812a3e942caa568831a5359459ebb63ccdb6b1ae1c9fbd4ecfeac4abbcdd Jul 07 20:11:50.207906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:50.207851 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" event={"ID":"9b6b01a9-25ce-4982-ad5d-d005c928122c","Type":"ContainerStarted","Data":"6786812a3e942caa568831a5359459ebb63ccdb6b1ae1c9fbd4ecfeac4abbcdd"} Jul 07 20:11:52.220095 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:52.220054 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" event={"ID":"9b6b01a9-25ce-4982-ad5d-d005c928122c","Type":"ContainerStarted","Data":"50bc85ac68c25a59caebd9b06d5f100712f0e5f9efcdace31b41fe5903333d0b"} Jul 07 20:11:52.239527 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:52.239475 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" podStartSLOduration=1.92316639 podStartE2EDuration="3.239461696s" totalImagesPullingTime="1.316295306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:11:52.238028659 +0000 UTC m=+664.746701232" watchObservedRunningTime="2026-07-07 20:11:52.239461696 +0000 UTC m=+664.748134269" Jul 07 20:11:59.790955 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:59.790902 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:59.790955 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:59.790977 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:11:59.803471 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:11:59.803441 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:12:00.271839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:00.271809 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-77ljn" Jul 07 20:12:06.866398 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.866366 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v"] Jul 07 20:12:06.870078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.870061 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:06.872700 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.872679 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 20:12:06.880492 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.880471 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v"] Jul 07 20:12:06.887760 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.887739 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:06.970484 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.970454 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:06.970659 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.970510 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:06.970659 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.970537 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:06.970659 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.970557 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3606f8af-a164-4a62-815e-9ee64f98eb21-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:06.970659 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.970594 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnvdk\" (UniqueName: \"kubernetes.io/projected/3606f8af-a164-4a62-815e-9ee64f98eb21-kube-api-access-tnvdk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:06.970659 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:06.970615 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.071923 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.071888 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnvdk\" (UniqueName: \"kubernetes.io/projected/3606f8af-a164-4a62-815e-9ee64f98eb21-kube-api-access-tnvdk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.071923 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.071926 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.072213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.071980 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.072213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.072035 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.072213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.072055 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.072213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.072073 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3606f8af-a164-4a62-815e-9ee64f98eb21-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.072391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.072342 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.072435 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.072402 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.072474 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.072459 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.074352 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.074325 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3606f8af-a164-4a62-815e-9ee64f98eb21-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.074592 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.074575 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3606f8af-a164-4a62-815e-9ee64f98eb21-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.079963 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.079942 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnvdk\" (UniqueName: \"kubernetes.io/projected/3606f8af-a164-4a62-815e-9ee64f98eb21-kube-api-access-tnvdk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-65j9v\" (UID: \"3606f8af-a164-4a62-815e-9ee64f98eb21\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.180899 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.180818 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:07.319498 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:07.319472 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v"] Jul 07 20:12:07.321215 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:12:07.321187 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3606f8af_a164_4a62_815e_9ee64f98eb21.slice/crio-dd59c262c1fc5b8b4043ff05f789b85aed210bdc2547efbed093effdc93eb99b WatchSource:0}: Error finding container dd59c262c1fc5b8b4043ff05f789b85aed210bdc2547efbed093effdc93eb99b: Status 404 returned error can't find the container with id dd59c262c1fc5b8b4043ff05f789b85aed210bdc2547efbed093effdc93eb99b Jul 07 20:12:08.305065 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:08.305023 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" event={"ID":"3606f8af-a164-4a62-815e-9ee64f98eb21","Type":"ContainerStarted","Data":"e6fb15ee8b15184d2942e7b57c0167616ffb5f697e1479ed44be44241e1ffee1"} Jul 07 20:12:08.305550 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:08.305093 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" event={"ID":"3606f8af-a164-4a62-815e-9ee64f98eb21","Type":"ContainerStarted","Data":"dd59c262c1fc5b8b4043ff05f789b85aed210bdc2547efbed093effdc93eb99b"} Jul 07 20:12:08.325357 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:08.325301 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" podStartSLOduration=2.089809085 podStartE2EDuration="2.325285629s" totalImagesPullingTime="235.476544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:12:08.323778936 +0000 UTC m=+680.832451512" watchObservedRunningTime="2026-07-07 20:12:08.325285629 +0000 UTC m=+680.833958413" Jul 07 20:12:09.283458 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:09.283425 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:09.591731 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:09.591638 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:09.986710 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:09.986676 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:10.086361 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:10.086326 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:17.181562 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:17.181526 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:17.181957 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:17.181589 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:17.194148 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:17.194118 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:17.362429 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:17.362403 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-65j9v" Jul 07 20:12:18.191793 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:18.191737 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:34.362724 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.362691 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b"] Jul 07 20:12:34.368912 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.368893 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.371771 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.371748 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 07 20:12:34.376822 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.376797 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b"] Jul 07 20:12:34.391735 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.391703 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:34.395716 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.395693 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.395805 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.395757 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.395805 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.395783 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.395928 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.395807 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmqvk\" (UniqueName: \"kubernetes.io/projected/6f8be59f-98ca-4ab3-afa8-97448e39fedb-kube-api-access-nmqvk\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.395928 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.395866 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8be59f-98ca-4ab3-afa8-97448e39fedb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.395928 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.395893 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.496680 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.496650 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.496814 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.496705 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.496814 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.496748 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.496814 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.496775 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmqvk\" (UniqueName: \"kubernetes.io/projected/6f8be59f-98ca-4ab3-afa8-97448e39fedb-kube-api-access-nmqvk\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.497102 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.496817 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8be59f-98ca-4ab3-afa8-97448e39fedb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.497102 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.496861 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.497252 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.497228 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.497321 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.497276 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.497373 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.497324 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.500484 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.500461 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f8be59f-98ca-4ab3-afa8-97448e39fedb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.505252 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.505223 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f8be59f-98ca-4ab3-afa8-97448e39fedb-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.506628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.506603 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmqvk\" (UniqueName: \"kubernetes.io/projected/6f8be59f-98ca-4ab3-afa8-97448e39fedb-kube-api-access-nmqvk\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b\" (UID: \"6f8be59f-98ca-4ab3-afa8-97448e39fedb\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.679608 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.679521 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:34.816901 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:34.816866 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b"] Jul 07 20:12:34.818734 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:12:34.818700 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f8be59f_98ca_4ab3_afa8_97448e39fedb.slice/crio-f8cbf636584ae585c9ca9731701bb89f9da5c0d1621689caca3d99bf2ff21b64 WatchSource:0}: Error finding container f8cbf636584ae585c9ca9731701bb89f9da5c0d1621689caca3d99bf2ff21b64: Status 404 returned error can't find the container with id f8cbf636584ae585c9ca9731701bb89f9da5c0d1621689caca3d99bf2ff21b64 Jul 07 20:12:35.450951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:35.450852 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" event={"ID":"6f8be59f-98ca-4ab3-afa8-97448e39fedb","Type":"ContainerStarted","Data":"3eea3f0ebd5d12bbcbb1db681ca1e94a7247cf4af1d899e661f96a50df8e7b83"} Jul 07 20:12:35.450951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:35.450906 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" event={"ID":"6f8be59f-98ca-4ab3-afa8-97448e39fedb","Type":"ContainerStarted","Data":"f8cbf636584ae585c9ca9731701bb89f9da5c0d1621689caca3d99bf2ff21b64"} Jul 07 20:12:35.471098 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:35.471054 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" podStartSLOduration=1.225292342 podStartE2EDuration="1.471038689s" totalImagesPullingTime="245.746347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:12:35.47068151 +0000 UTC m=+707.979354087" watchObservedRunningTime="2026-07-07 20:12:35.471038689 +0000 UTC m=+707.979711265" Jul 07 20:12:35.924901 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:35.924870 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:12:44.679902 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:44.679818 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:44.679902 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:44.679889 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:44.692335 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:44.692313 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:45.521027 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:45.520975 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b" Jul 07 20:12:54.276940 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:12:54.276902 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:13:30.491800 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:30.491763 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:13:41.138131 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.138094 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bc46d547b-rmn8k"] Jul 07 20:13:41.141552 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.141533 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.149404 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.149367 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bc46d547b-rmn8k"] Jul 07 20:13:41.204326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.204291 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1487b873-3aea-4003-92bc-830301deafaf-tls-cert\") pod \"authorino-5bc46d547b-rmn8k\" (UID: \"1487b873-3aea-4003-92bc-830301deafaf\") " pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.204506 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.204342 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxvcb\" (UniqueName: \"kubernetes.io/projected/1487b873-3aea-4003-92bc-830301deafaf-kube-api-access-mxvcb\") pod \"authorino-5bc46d547b-rmn8k\" (UID: \"1487b873-3aea-4003-92bc-830301deafaf\") " pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.305390 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.305344 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1487b873-3aea-4003-92bc-830301deafaf-tls-cert\") pod \"authorino-5bc46d547b-rmn8k\" (UID: \"1487b873-3aea-4003-92bc-830301deafaf\") " pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.305577 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.305405 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxvcb\" (UniqueName: \"kubernetes.io/projected/1487b873-3aea-4003-92bc-830301deafaf-kube-api-access-mxvcb\") pod \"authorino-5bc46d547b-rmn8k\" (UID: \"1487b873-3aea-4003-92bc-830301deafaf\") " pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.307892 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.307870 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1487b873-3aea-4003-92bc-830301deafaf-tls-cert\") pod \"authorino-5bc46d547b-rmn8k\" (UID: \"1487b873-3aea-4003-92bc-830301deafaf\") " pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.314550 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.314526 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxvcb\" (UniqueName: \"kubernetes.io/projected/1487b873-3aea-4003-92bc-830301deafaf-kube-api-access-mxvcb\") pod \"authorino-5bc46d547b-rmn8k\" (UID: \"1487b873-3aea-4003-92bc-830301deafaf\") " pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.428497 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.428407 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:13:41.451960 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.451929 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bc46d547b-rmn8k" Jul 07 20:13:41.525779 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.525738 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:13:41.588045 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.588018 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bc46d547b-rmn8k"] Jul 07 20:13:41.589817 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:13:41.589789 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1487b873_3aea_4003_92bc_830301deafaf.slice/crio-a8bec651958a93ae634a0cd87d5c325fe6852c53dd59983582247f18b706e6a2 WatchSource:0}: Error finding container a8bec651958a93ae634a0cd87d5c325fe6852c53dd59983582247f18b706e6a2: Status 404 returned error can't find the container with id a8bec651958a93ae634a0cd87d5c325fe6852c53dd59983582247f18b706e6a2 Jul 07 20:13:41.813060 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:41.813026 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bc46d547b-rmn8k" event={"ID":"1487b873-3aea-4003-92bc-830301deafaf","Type":"ContainerStarted","Data":"a8bec651958a93ae634a0cd87d5c325fe6852c53dd59983582247f18b706e6a2"} Jul 07 20:13:42.676822 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:42.676779 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:13:42.775966 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:42.775916 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:13:42.831830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:42.831781 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bc46d547b-rmn8k" event={"ID":"1487b873-3aea-4003-92bc-830301deafaf","Type":"ContainerStarted","Data":"7da3f8cd5930b0d0caec6f551e8dcce511f2e729add773720a5c0fc1ec8fb375"} Jul 07 20:13:42.861899 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:42.861844 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bc46d547b-rmn8k" podStartSLOduration=1.282604538 podStartE2EDuration="1.8618278s" totalImagesPullingTime="579.223262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:13:42.860253072 +0000 UTC m=+775.368925658" watchObservedRunningTime="2026-07-07 20:13:42.8618278 +0000 UTC m=+775.370500384" Jul 07 20:13:42.897360 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:42.897301 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79f897cdbf-dhf2m"] Jul 07 20:13:42.897645 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:42.897568 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" podUID="00a6dc33-e909-44b3-b302-5af5067127d8" containerName="authorino" containerID="cri-o://a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d" gracePeriod=30 Jul 07 20:13:43.163727 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.163700 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:13:43.327403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.327360 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00a6dc33-e909-44b3-b302-5af5067127d8-tls-cert\") pod \"00a6dc33-e909-44b3-b302-5af5067127d8\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " Jul 07 20:13:43.327403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.327408 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrvsn\" (UniqueName: \"kubernetes.io/projected/00a6dc33-e909-44b3-b302-5af5067127d8-kube-api-access-wrvsn\") pod \"00a6dc33-e909-44b3-b302-5af5067127d8\" (UID: \"00a6dc33-e909-44b3-b302-5af5067127d8\") " Jul 07 20:13:43.329575 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.329544 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a6dc33-e909-44b3-b302-5af5067127d8-kube-api-access-wrvsn" (OuterVolumeSpecName: "kube-api-access-wrvsn") pod "00a6dc33-e909-44b3-b302-5af5067127d8" (UID: "00a6dc33-e909-44b3-b302-5af5067127d8"). InnerVolumeSpecName "kube-api-access-wrvsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:13:43.338273 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.338245 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a6dc33-e909-44b3-b302-5af5067127d8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "00a6dc33-e909-44b3-b302-5af5067127d8" (UID: "00a6dc33-e909-44b3-b302-5af5067127d8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 20:13:43.428560 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.428521 2553 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00a6dc33-e909-44b3-b302-5af5067127d8-tls-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:13:43.428560 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.428557 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrvsn\" (UniqueName: \"kubernetes.io/projected/00a6dc33-e909-44b3-b302-5af5067127d8-kube-api-access-wrvsn\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:13:43.844229 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.844183 2553 generic.go:356] "Generic (PLEG): container finished" podID="00a6dc33-e909-44b3-b302-5af5067127d8" containerID="a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d" exitCode=0 Jul 07 20:13:43.844229 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.844221 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" Jul 07 20:13:43.844734 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.844260 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" event={"ID":"00a6dc33-e909-44b3-b302-5af5067127d8","Type":"ContainerDied","Data":"a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d"} Jul 07 20:13:43.844734 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.844304 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79f897cdbf-dhf2m" event={"ID":"00a6dc33-e909-44b3-b302-5af5067127d8","Type":"ContainerDied","Data":"928a8d0acf72e3e2c29ee643f2cf70c6fcf109a8becb0e196d1c12df8f25c491"} Jul 07 20:13:43.844734 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.844327 2553 scope.go:117] "RemoveContainer" containerID="a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d" Jul 07 20:13:43.854100 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.854075 2553 scope.go:117] "RemoveContainer" containerID="a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d" Jul 07 20:13:43.854384 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:13:43.854368 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d\": container with ID starting with a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d not found: ID does not exist" containerID="a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d" Jul 07 20:13:43.854439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.854393 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d"} err="failed to get container status \"a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d\": rpc error: code = NotFound desc = could not find container \"a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d\": container with ID starting with a03c35bde44144131f37217bee18e26927953966d081af5b444e096871606e1d not found: ID does not exist" Jul 07 20:13:43.868617 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.868577 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79f897cdbf-dhf2m"] Jul 07 20:13:43.874663 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:43.874635 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79f897cdbf-dhf2m"] Jul 07 20:13:44.029867 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:44.029823 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00a6dc33-e909-44b3-b302-5af5067127d8" path="/var/lib/kubelet/pods/00a6dc33-e909-44b3-b302-5af5067127d8/volumes" Jul 07 20:13:46.003054 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:46.003017 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:13:46.104681 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:13:46.104627 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:14:28.081689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:14:28.081646 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:14:28.152295 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:14:28.150929 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:14:54.261485 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:14:54.261453 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:14:55.614798 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:14:55.614763 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:33.439280 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:33.439248 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:36.521042 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:36.520986 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:36.622684 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:36.622648 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:36.766823 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:36.766776 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:42.992114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:42.991985 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:43.177204 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:43.177168 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:43.246747 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:43.246648 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:43.328012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:43.327951 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:47.992473 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:47.992437 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:15:47.993503 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:47.993485 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:15:49.506659 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:49.506622 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:15:49.568082 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:15:49.568047 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:17:54.520440 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:17:54.520400 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:18:27.957172 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:18:27.957129 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:19:47.261373 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:19:47.261336 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:20:48.029890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:20:48.029859 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:20:48.030787 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:20:48.030770 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:21:17.357692 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:21:17.357644 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:21:17.446603 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:21:17.446565 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:24:07.601599 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:24:07.601560 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:24:17.343497 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:24:17.343456 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:24:17.441383 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:24:17.441329 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:25:48.063416 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:25:48.063388 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:25:48.066191 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:25:48.066158 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:26:32.388545 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:32.388504 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:33.161071 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:33.161029 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:33.261901 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:33.261856 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:38.955280 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:38.955221 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:47.276125 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:47.276085 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:52.578649 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:52.578611 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:55.988158 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:55.988113 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:56.089467 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:56.089424 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:26:56.493951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:26:56.493916 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:02.723830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.723789 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:02.824827 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.824792 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:02.900745 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.900714 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz"] Jul 07 20:27:02.901114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.901101 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00a6dc33-e909-44b3-b302-5af5067127d8" containerName="authorino" Jul 07 20:27:02.901166 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.901115 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="00a6dc33-e909-44b3-b302-5af5067127d8" containerName="authorino" Jul 07 20:27:02.901206 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.901174 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="00a6dc33-e909-44b3-b302-5af5067127d8" containerName="authorino" Jul 07 20:27:02.904336 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.904317 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.907064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.907042 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a0845fbd-openshift-default-dockercfg-gkbtf\"" Jul 07 20:27:02.913240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913211 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-token\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913332 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913259 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-data\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913332 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913323 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-socket\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913420 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913355 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-podinfo\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913420 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913385 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-kube-api-access-54cfj\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913420 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913413 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-credential-socket\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913537 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913467 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-certs\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913537 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913499 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-envoy\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.913640 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.913562 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bfad5d-ff3f-4d7a-868e-00677042328a-istiod-ca-cert\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:02.930567 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.930535 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz"] Jul 07 20:27:02.931475 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:02.931454 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:03.013974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.013938 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-credential-socket\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014178 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014011 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-certs\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014178 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014037 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-envoy\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014178 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014063 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bfad5d-ff3f-4d7a-868e-00677042328a-istiod-ca-cert\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014178 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014164 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-token\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014404 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014190 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-data\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014404 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014273 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-socket\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014404 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014313 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-podinfo\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014404 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014348 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-kube-api-access-54cfj\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014400 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-credential-socket\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014593 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-certs\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014718 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014634 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-socket\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014718 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014660 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-data\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.014933 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.014911 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bfad5d-ff3f-4d7a-868e-00677042328a-istiod-ca-cert\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.016674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.016653 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-envoy\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.016886 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.016868 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-podinfo\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.024741 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.024691 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-token\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.024972 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.024931 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-kube-api-access-54cfj\") pod \"e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.025344 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.025289 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:03.215391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.215359 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:03.357490 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.357466 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz"] Jul 07 20:27:03.359405 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:27:03.359375 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bfad5d_ff3f_4d7a_868e_00677042328a.slice/crio-e06055b38dbecf56e2a476abfd3f2e8661a17dd42a70c9fbbc974cd5b7822d96 WatchSource:0}: Error finding container e06055b38dbecf56e2a476abfd3f2e8661a17dd42a70c9fbbc974cd5b7822d96: Status 404 returned error can't find the container with id e06055b38dbecf56e2a476abfd3f2e8661a17dd42a70c9fbbc974cd5b7822d96 Jul 07 20:27:03.364919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.364888 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:03.365012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.364951 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:03.365012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:03.364981 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:04.130009 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.129943 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:04.209710 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.209650 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:04.222697 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.222658 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" event={"ID":"99bfad5d-ff3f-4d7a-868e-00677042328a","Type":"ContainerStarted","Data":"c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9"} Jul 07 20:27:04.222867 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.222726 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" event={"ID":"99bfad5d-ff3f-4d7a-868e-00677042328a","Type":"ContainerStarted","Data":"e06055b38dbecf56e2a476abfd3f2e8661a17dd42a70c9fbbc974cd5b7822d96"} Jul 07 20:27:04.247780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.247729 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" podStartSLOduration=2.24771392 podStartE2EDuration="2.24771392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:27:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:27:04.244683377 +0000 UTC m=+1576.753355953" watchObservedRunningTime="2026-07-07 20:27:04.24771392 +0000 UTC m=+1576.756386494" Jul 07 20:27:04.282998 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.282974 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:04.384307 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.384220 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:04.704029 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.703932 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:04.804419 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:04.804386 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:05.216431 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:05.216389 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:05.221475 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:05.221448 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:05.229159 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:05.229106 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:05.230301 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:05.230280 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:07.231656 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:07.231608 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:08.815285 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:08.815246 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:09.267711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:09.267678 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:09.438792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:09.438758 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz"] Jul 07 20:27:09.439044 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:09.438984 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" podUID="99bfad5d-ff3f-4d7a-868e-00677042328a" containerName="istio-proxy" containerID="cri-o://c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9" gracePeriod=30 Jul 07 20:27:09.460931 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:09.460906 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:09.574326 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:09.574233 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:09.645117 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:09.645080 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:10.988762 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:10.988730 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:15.509571 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.509510 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:15.625342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625309 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-socket\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625348 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-credential-socket\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625365 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-certs\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625396 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-token\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625448 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-podinfo\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625484 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-data\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625500 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-kube-api-access-54cfj\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625530 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-envoy\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.625593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.625548 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bfad5d-ff3f-4d7a-868e-00677042328a-istiod-ca-cert\") pod \"99bfad5d-ff3f-4d7a-868e-00677042328a\" (UID: \"99bfad5d-ff3f-4d7a-868e-00677042328a\") " Jul 07 20:27:15.627221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.627198 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:15.629147 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.629105 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:15.629324 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.629257 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:15.629417 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.629395 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99bfad5d-ff3f-4d7a-868e-00677042328a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:27:15.629889 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.629865 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-data" (OuterVolumeSpecName: "istio-data") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:15.631354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.631333 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:27:15.631354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.631340 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:15.631626 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.631605 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-kube-api-access-54cfj" (OuterVolumeSpecName: "kube-api-access-54cfj") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "kube-api-access-54cfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:27:15.631679 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.631627 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-token" (OuterVolumeSpecName: "istio-token") pod "99bfad5d-ff3f-4d7a-868e-00677042328a" (UID: "99bfad5d-ff3f-4d7a-868e-00677042328a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:27:15.727097 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727060 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727097 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727088 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54cfj\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-kube-api-access-54cfj\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727097 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727098 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727097 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727107 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99bfad5d-ff3f-4d7a-868e-00677042328a-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727356 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727116 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727356 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727124 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727356 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727133 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99bfad5d-ff3f-4d7a-868e-00677042328a-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727356 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727144 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.727356 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.727151 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99bfad5d-ff3f-4d7a-868e-00677042328a-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:15.813873 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.813790 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:15.876288 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.876251 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:15.989443 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.989409 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz"] Jul 07 20:27:15.990028 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.990009 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99bfad5d-ff3f-4d7a-868e-00677042328a" containerName="istio-proxy" Jul 07 20:27:15.990084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.990032 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="99bfad5d-ff3f-4d7a-868e-00677042328a" containerName="istio-proxy" Jul 07 20:27:15.990164 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.990152 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="99bfad5d-ff3f-4d7a-868e-00677042328a" containerName="istio-proxy" Jul 07 20:27:15.993939 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.993922 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:15.996616 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:15.996596 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-557e05f1-openshift-default-dockercfg-f2m2f\"" Jul 07 20:27:16.004516 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.004491 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz"] Jul 07 20:27:16.024861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.024821 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:16.030336 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030309 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-socket\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030346 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-credential-socket\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030389 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-certs\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030413 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-podinfo\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030440 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istiod-ca-cert\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030625 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030510 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-data\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030625 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030535 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrzfp\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-kube-api-access-zrzfp\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030625 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030570 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-token\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.030625 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.030605 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-envoy\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.125567 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.125478 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:16.131623 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131596 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-envoy\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.131711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131659 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-socket\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.131711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131678 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-credential-socket\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.131815 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131794 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-certs\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.131860 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-podinfo\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.131913 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131869 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istiod-ca-cert\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.131970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131926 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-data\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.131970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.131958 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrzfp\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-kube-api-access-zrzfp\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.132103 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.132059 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-token\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.132103 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.132088 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-socket\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.132263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.132236 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-certs\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.132516 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.132490 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-credential-socket\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.132638 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.132516 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-data\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.132638 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.132616 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istiod-ca-cert\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.134069 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.134049 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-envoy\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.134292 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.134275 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-podinfo\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.140758 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.140734 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-token\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.141146 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.141128 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrzfp\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-kube-api-access-zrzfp\") pod \"e2e-ait-557e05f1-openshift-default-8fbddd7-945tz\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.288551 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.288521 2553 generic.go:356] "Generic (PLEG): container finished" podID="99bfad5d-ff3f-4d7a-868e-00677042328a" containerID="c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9" exitCode=0 Jul 07 20:27:16.288748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.288567 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" Jul 07 20:27:16.288748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.288617 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" event={"ID":"99bfad5d-ff3f-4d7a-868e-00677042328a","Type":"ContainerDied","Data":"c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9"} Jul 07 20:27:16.288748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.288676 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz" event={"ID":"99bfad5d-ff3f-4d7a-868e-00677042328a","Type":"ContainerDied","Data":"e06055b38dbecf56e2a476abfd3f2e8661a17dd42a70c9fbbc974cd5b7822d96"} Jul 07 20:27:16.288748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.288707 2553 scope.go:117] "RemoveContainer" containerID="c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9" Jul 07 20:27:16.297843 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.297828 2553 scope.go:117] "RemoveContainer" containerID="c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9" Jul 07 20:27:16.298097 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:27:16.298081 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9\": container with ID starting with c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9 not found: ID does not exist" containerID="c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9" Jul 07 20:27:16.298150 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.298104 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9"} err="failed to get container status \"c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9\": rpc error: code = NotFound desc = could not find container \"c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9\": container with ID starting with c5975b2868e8fccee8dd59f1c953ad2ce08df9914e4b21050816c7f718290ab9 not found: ID does not exist" Jul 07 20:27:16.306053 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.306031 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:16.312962 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.312936 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz"] Jul 07 20:27:16.318919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.318899 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a0845fbd-openshift-default-7f74fccf99-whpgz"] Jul 07 20:27:16.456140 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.456106 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz"] Jul 07 20:27:16.457005 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:27:16.456961 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07905cda_6a98_43f9_a18e_da2ecf62d9cc.slice/crio-3a5e6ed1f6ea4fb812200464ee383982ee63c3b2c66b6beb04080c21e834f9c3 WatchSource:0}: Error finding container 3a5e6ed1f6ea4fb812200464ee383982ee63c3b2c66b6beb04080c21e834f9c3: Status 404 returned error can't find the container with id 3a5e6ed1f6ea4fb812200464ee383982ee63c3b2c66b6beb04080c21e834f9c3 Jul 07 20:27:16.459286 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.459257 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:16.459357 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.459319 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:16.459357 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:16.459348 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:17.134629 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.134592 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:17.217829 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.217775 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:17.284429 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.284391 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:17.296939 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.296905 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" event={"ID":"07905cda-6a98-43f9-a18e-da2ecf62d9cc","Type":"ContainerStarted","Data":"6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97"} Jul 07 20:27:17.297116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.297019 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" event={"ID":"07905cda-6a98-43f9-a18e-da2ecf62d9cc","Type":"ContainerStarted","Data":"3a5e6ed1f6ea4fb812200464ee383982ee63c3b2c66b6beb04080c21e834f9c3"} Jul 07 20:27:17.306712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.306695 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:17.323670 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.323606 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" podStartSLOduration=2.323590148 podStartE2EDuration="2.323590148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:27:17.321930086 +0000 UTC m=+1589.830602661" watchObservedRunningTime="2026-07-07 20:27:17.323590148 +0000 UTC m=+1589.832262724" Jul 07 20:27:17.336549 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.336521 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:17.485633 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.485590 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:17.890130 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.890040 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:17.989904 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:17.989868 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:18.030646 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:18.030613 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99bfad5d-ff3f-4d7a-868e-00677042328a" path="/var/lib/kubelet/pods/99bfad5d-ff3f-4d7a-868e-00677042328a/volumes" Jul 07 20:27:18.303130 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:18.303097 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:18.303948 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:18.303930 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:19.083007 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:19.082957 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:22.357567 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:22.357530 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:22.699757 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:22.699679 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:28.530745 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.530718 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz"] Jul 07 20:27:28.531194 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.530915 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" podUID="07905cda-6a98-43f9-a18e-da2ecf62d9cc" containerName="istio-proxy" containerID="cri-o://6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97" gracePeriod=30 Jul 07 20:27:28.553215 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.553139 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:28.677563 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.677532 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:28.678580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.678563 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:28.761270 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761241 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-data\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.761417 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761312 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istiod-ca-cert\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.761417 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761392 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-socket\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.761530 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761420 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-podinfo\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.761530 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761444 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-envoy\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.761637 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761585 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-credential-socket\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.761763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761738 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:27:28.761867 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.761844 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrzfp\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-kube-api-access-zrzfp\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.762393 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.762119 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:28.762393 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.762196 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:28.762563 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.762330 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:28.762691 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.762666 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:28.762818 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.762470 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-certs\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.763627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.763055 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-token\") pod \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\" (UID: \"07905cda-6a98-43f9-a18e-da2ecf62d9cc\") " Jul 07 20:27:28.764395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764363 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-kube-api-access-zrzfp" (OuterVolumeSpecName: "kube-api-access-zrzfp") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "kube-api-access-zrzfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:27:28.764493 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764467 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:27:28.764648 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764628 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.764728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764653 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.764728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764667 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.764728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764681 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrzfp\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-kube-api-access-zrzfp\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.764728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764694 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.764728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764707 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.764728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764718 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.765051 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.764723 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:28.766182 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.766152 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "07905cda-6a98-43f9-a18e-da2ecf62d9cc" (UID: "07905cda-6a98-43f9-a18e-da2ecf62d9cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:27:28.865402 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.865371 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:28.865402 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:28.865396 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07905cda-6a98-43f9-a18e-da2ecf62d9cc-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:29.377330 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.377292 2553 generic.go:356] "Generic (PLEG): container finished" podID="07905cda-6a98-43f9-a18e-da2ecf62d9cc" containerID="6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97" exitCode=0 Jul 07 20:27:29.377505 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.377354 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" Jul 07 20:27:29.377505 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.377378 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" event={"ID":"07905cda-6a98-43f9-a18e-da2ecf62d9cc","Type":"ContainerDied","Data":"6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97"} Jul 07 20:27:29.377505 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.377449 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz" event={"ID":"07905cda-6a98-43f9-a18e-da2ecf62d9cc","Type":"ContainerDied","Data":"3a5e6ed1f6ea4fb812200464ee383982ee63c3b2c66b6beb04080c21e834f9c3"} Jul 07 20:27:29.377505 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.377471 2553 scope.go:117] "RemoveContainer" containerID="6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97" Jul 07 20:27:29.386807 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.386790 2553 scope.go:117] "RemoveContainer" containerID="6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97" Jul 07 20:27:29.387080 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:27:29.387056 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97\": container with ID starting with 6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97 not found: ID does not exist" containerID="6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97" Jul 07 20:27:29.387150 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.387093 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97"} err="failed to get container status \"6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97\": rpc error: code = NotFound desc = could not find container \"6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97\": container with ID starting with 6fe0a9d32f5e033c428e888f9482f5b043d1aef2aa291fc46250a3907f6f2f97 not found: ID does not exist" Jul 07 20:27:29.400073 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.400053 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz"] Jul 07 20:27:29.403864 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:29.403841 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-557e05f1-openshift-default-8fbddd7-945tz"] Jul 07 20:27:30.030365 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:30.030336 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07905cda-6a98-43f9-a18e-da2ecf62d9cc" path="/var/lib/kubelet/pods/07905cda-6a98-43f9-a18e-da2ecf62d9cc/volumes" Jul 07 20:27:30.659654 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:30.659625 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:34.555873 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.555839 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:34.653746 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.653708 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:34.752266 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.752237 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr"] Jul 07 20:27:34.752660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.752646 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07905cda-6a98-43f9-a18e-da2ecf62d9cc" containerName="istio-proxy" Jul 07 20:27:34.752708 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.752662 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="07905cda-6a98-43f9-a18e-da2ecf62d9cc" containerName="istio-proxy" Jul 07 20:27:34.752747 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.752737 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="07905cda-6a98-43f9-a18e-da2ecf62d9cc" containerName="istio-proxy" Jul 07 20:27:34.757106 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.757083 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.760628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.760604 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-70f9ecd0-openshift-default-dockercfg-cwfnf\"" Jul 07 20:27:34.768800 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.768775 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr"] Jul 07 20:27:34.790324 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.790295 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:34.887323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.887238 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:34.935732 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935704 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-envoy\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.935877 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935739 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-credential-socket\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.935877 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935757 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-data\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.935877 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935817 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-certs\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.935877 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-token\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.935877 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935867 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqvt4\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-kube-api-access-pqvt4\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.936072 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935893 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-podinfo\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.936072 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.935913 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-socket\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:34.936072 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:34.936012 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istiod-ca-cert\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037390 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037349 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-envoy\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037556 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037399 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-credential-socket\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037556 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037427 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-data\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037556 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037480 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-certs\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037556 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037518 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-token\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037556 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037549 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqvt4\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-kube-api-access-pqvt4\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037809 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037589 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-podinfo\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037809 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037619 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-socket\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037809 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037664 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istiod-ca-cert\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037882 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-credential-socket\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037911 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-data\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.037959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.037941 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-certs\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.038186 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.038158 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-socket\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.038371 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.038355 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istiod-ca-cert\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.039719 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.039688 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-envoy\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.040047 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.040030 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-podinfo\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.047060 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.047033 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-token\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.047409 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.047389 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqvt4\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-kube-api-access-pqvt4\") pod \"e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.070176 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.070139 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:35.216725 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.216691 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr"] Jul 07 20:27:35.220478 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:27:35.220449 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fb37373_9437_4ac1_8d53_936f6ec4b63f.slice/crio-fb5d4b61c20c2596e1604da8fb35b58205c9a8f6ddf1d395792c4497389241b1 WatchSource:0}: Error finding container fb5d4b61c20c2596e1604da8fb35b58205c9a8f6ddf1d395792c4497389241b1: Status 404 returned error can't find the container with id fb5d4b61c20c2596e1604da8fb35b58205c9a8f6ddf1d395792c4497389241b1 Jul 07 20:27:35.222602 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.222567 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:35.222674 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.222658 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:35.222720 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.222700 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:27:35.413810 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.413721 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" event={"ID":"1fb37373-9437-4ac1-8d53-936f6ec4b63f","Type":"ContainerStarted","Data":"5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55"} Jul 07 20:27:35.413810 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.413780 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" event={"ID":"1fb37373-9437-4ac1-8d53-936f6ec4b63f","Type":"ContainerStarted","Data":"fb5d4b61c20c2596e1604da8fb35b58205c9a8f6ddf1d395792c4497389241b1"} Jul 07 20:27:35.437635 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.437577 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" podStartSLOduration=1.437558042 podStartE2EDuration="1.437558042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:27:35.435966854 +0000 UTC m=+1607.944639434" watchObservedRunningTime="2026-07-07 20:27:35.437558042 +0000 UTC m=+1607.946230618" Jul 07 20:27:35.944128 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:35.944092 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:36.025348 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.025312 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:36.071072 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.071039 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:36.075610 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.075588 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:36.088922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.088897 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:36.188636 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.188608 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:36.419040 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.418986 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:36.419913 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.419893 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:36.477954 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.477917 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:36.579363 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:36.579328 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:38.239078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:38.239036 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:40.418660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:40.418566 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:40.581345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:40.581313 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr"] Jul 07 20:27:40.581585 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:40.581515 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" podUID="1fb37373-9437-4ac1-8d53-936f6ec4b63f" containerName="istio-proxy" containerID="cri-o://5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55" gracePeriod=30 Jul 07 20:27:40.605706 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:40.605673 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:40.709278 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:40.709176 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:40.802041 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:40.802008 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:42.167597 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:42.167562 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:46.742967 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.742908 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:46.857788 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.857759 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqvt4\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-kube-api-access-pqvt4\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.857788 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.857794 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-credential-socket\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.857819 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-data\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.857847 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-socket\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.857880 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-certs\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.857900 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istiod-ca-cert\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.857932 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-token\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858052 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-envoy\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858078 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858082 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-podinfo\") pod \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\" (UID: \"1fb37373-9437-4ac1-8d53-936f6ec4b63f\") " Jul 07 20:27:46.858445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858098 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:46.858445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858278 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:46.858445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858417 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.858445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858436 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.858651 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858495 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:27:46.858651 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858605 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:46.858756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.858710 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:46.860161 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.860127 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:27:46.860161 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.860131 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-kube-api-access-pqvt4" (OuterVolumeSpecName: "kube-api-access-pqvt4") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "kube-api-access-pqvt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:27:46.860303 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.860289 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:27:46.860350 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.860336 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fb37373-9437-4ac1-8d53-936f6ec4b63f" (UID: "1fb37373-9437-4ac1-8d53-936f6ec4b63f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:27:46.951921 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.951886 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:46.959051 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.959029 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqvt4\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-kube-api-access-pqvt4\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.959127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.959053 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.959127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.959063 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.959127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.959071 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.959127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.959079 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.959127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.959087 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:46.959127 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:46.959094 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fb37373-9437-4ac1-8d53-936f6ec4b63f-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:27:47.051946 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.051862 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:47.462970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.462876 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:47.487220 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.487190 2553 generic.go:356] "Generic (PLEG): container finished" podID="1fb37373-9437-4ac1-8d53-936f6ec4b63f" containerID="5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55" exitCode=0 Jul 07 20:27:47.487383 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.487240 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" Jul 07 20:27:47.487383 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.487286 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" event={"ID":"1fb37373-9437-4ac1-8d53-936f6ec4b63f","Type":"ContainerDied","Data":"5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55"} Jul 07 20:27:47.487383 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.487346 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr" event={"ID":"1fb37373-9437-4ac1-8d53-936f6ec4b63f","Type":"ContainerDied","Data":"fb5d4b61c20c2596e1604da8fb35b58205c9a8f6ddf1d395792c4497389241b1"} Jul 07 20:27:47.487383 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.487374 2553 scope.go:117] "RemoveContainer" containerID="5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55" Jul 07 20:27:47.497461 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.497445 2553 scope.go:117] "RemoveContainer" containerID="5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55" Jul 07 20:27:47.497716 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:27:47.497696 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55\": container with ID starting with 5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55 not found: ID does not exist" containerID="5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55" Jul 07 20:27:47.497761 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.497725 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55"} err="failed to get container status \"5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55\": rpc error: code = NotFound desc = could not find container \"5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55\": container with ID starting with 5e1ba7f8633b2df2a6e227c28e7dc95b0f7217dfc44052cb3a351e26b834ce55 not found: ID does not exist" Jul 07 20:27:47.513184 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.513164 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr"] Jul 07 20:27:47.517807 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.517788 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-70f9ecd0-openshift-default-69c79c9cff-lsxnr"] Jul 07 20:27:47.523067 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.523036 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:47.584533 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:47.584511 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:48.030581 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:48.030550 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fb37373-9437-4ac1-8d53-936f6ec4b63f" path="/var/lib/kubelet/pods/1fb37373-9437-4ac1-8d53-936f6ec4b63f/volumes" Jul 07 20:27:49.195166 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:49.195126 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:51.687319 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:51.687287 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:55.307319 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:55.307273 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:55.407949 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:55.407909 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:55.507893 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:55.507852 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:55.606160 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:55.606078 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:27:56.076941 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:27:56.076909 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:22.131291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:22.131255 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:28.265085 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:28.265044 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:28.372819 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:28.372786 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:28.465034 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:28.464983 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:44.553443 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:44.553406 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:50.863395 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:50.863353 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:50.926615 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:50.926580 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:51.037700 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:51.037658 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:28:51.138541 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:28:51.138448 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:01.667245 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:01.667203 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:02.514107 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:02.514070 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:08.500793 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:08.500764 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:08.557345 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:08.557293 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:08.640919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:08.640887 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:08.740958 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:08.740923 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:08.843476 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:08.843386 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:09.650005 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:09.649955 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:10.737504 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:10.737418 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:11.585847 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:11.585812 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:17.703621 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:17.703582 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:17.803516 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:17.803483 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:18.429192 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:18.429146 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:24.639197 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:24.639163 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:24.699627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:24.699590 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:24.906968 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:24.906888 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:25.213651 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.213573 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4"] Jul 07 20:29:25.214000 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.213972 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fb37373-9437-4ac1-8d53-936f6ec4b63f" containerName="istio-proxy" Jul 07 20:29:25.214070 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.214003 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fb37373-9437-4ac1-8d53-936f6ec4b63f" containerName="istio-proxy" Jul 07 20:29:25.214113 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.214099 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fb37373-9437-4ac1-8d53-936f6ec4b63f" containerName="istio-proxy" Jul 07 20:29:25.217187 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.217172 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.219900 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.219875 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e8652739-openshift-default-dockercfg-49v6k\"" Jul 07 20:29:25.230529 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.230506 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:25.238214 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.238186 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4"] Jul 07 20:29:25.336932 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.336895 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-data\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337128 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337021 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-istio-token\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337128 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337081 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d156cd-6fda-4747-8188-e0913da1e45c-istio-podinfo\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337128 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337101 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-socket\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337128 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337125 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhbbt\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-kube-api-access-xhbbt\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337154 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-certs\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337234 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-credential-socket\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337251 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-envoy\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.337300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.337273 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d156cd-6fda-4747-8188-e0913da1e45c-istiod-ca-cert\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.418544 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.418496 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:25.438698 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438669 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-istio-token\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.438838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438710 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d156cd-6fda-4747-8188-e0913da1e45c-istio-podinfo\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.438838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438729 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-socket\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.438838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438760 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhbbt\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-kube-api-access-xhbbt\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.438838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438797 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-certs\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438839 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-credential-socket\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438861 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-envoy\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438889 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d156cd-6fda-4747-8188-e0913da1e45c-istiod-ca-cert\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.438930 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-data\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439330 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.439263 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-socket\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439424 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.439399 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-certs\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439541 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.439419 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-credential-socket\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439541 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.439509 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-data\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.439730 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.439714 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d156cd-6fda-4747-8188-e0913da1e45c-istiod-ca-cert\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.441192 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.441175 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-envoy\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.441255 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.441238 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d156cd-6fda-4747-8188-e0913da1e45c-istio-podinfo\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.451754 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.451730 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-istio-token\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.451844 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.451793 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhbbt\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-kube-api-access-xhbbt\") pod \"e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.527329 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.527301 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:25.666697 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.666671 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4"] Jul 07 20:29:25.668418 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:29:25.668392 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d156cd_6fda_4747_8188_e0913da1e45c.slice/crio-5dc5f4270028d510f925ce218765aab7e3b602af53af0b2c3ab935f6ce8fad87 WatchSource:0}: Error finding container 5dc5f4270028d510f925ce218765aab7e3b602af53af0b2c3ab935f6ce8fad87: Status 404 returned error can't find the container with id 5dc5f4270028d510f925ce218765aab7e3b602af53af0b2c3ab935f6ce8fad87 Jul 07 20:29:25.670547 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.670465 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:29:25.670547 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.670532 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:29:25.670654 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:25.670563 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:29:26.029334 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:26.029298 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" event={"ID":"00d156cd-6fda-4747-8188-e0913da1e45c","Type":"ContainerStarted","Data":"13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8"} Jul 07 20:29:26.029506 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:26.029351 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" event={"ID":"00d156cd-6fda-4747-8188-e0913da1e45c","Type":"ContainerStarted","Data":"5dc5f4270028d510f925ce218765aab7e3b602af53af0b2c3ab935f6ce8fad87"} Jul 07 20:29:26.096256 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:26.096203 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" podStartSLOduration=1.096186496 podStartE2EDuration="1.096186496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:29:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:29:26.092910719 +0000 UTC m=+1718.601583293" watchObservedRunningTime="2026-07-07 20:29:26.096186496 +0000 UTC m=+1718.604859070" Jul 07 20:29:26.528413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:26.528385 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:26.533239 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:26.533216 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:27.031868 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:27.031834 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:27.032836 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:27.032815 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:28.210476 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:28.210442 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:28.302112 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:28.302079 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:28.363959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:28.363924 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:28.638274 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:28.638240 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:28.943530 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:28.943442 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:29.043627 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:29.043585 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:32.763167 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:32.763130 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:33.464740 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:33.464694 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:33.564919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:33.564883 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:33.916749 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:33.916716 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:40.388495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:40.388448 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:40.446440 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:40.446404 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:40.817011 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:40.816947 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:41.178926 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:41.178824 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:41.279457 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:41.279418 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:42.033574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.033492 2553 patch_prober.go:28] interesting pod/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 07 20:29:42.033574 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.033558 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 07 20:29:42.116753 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.116718 2553 generic.go:356] "Generic (PLEG): container finished" podID="00d156cd-6fda-4747-8188-e0913da1e45c" containerID="13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8" exitCode=0 Jul 07 20:29:42.116915 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.116796 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" event={"ID":"00d156cd-6fda-4747-8188-e0913da1e45c","Type":"ContainerDied","Data":"13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8"} Jul 07 20:29:42.117284 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.117266 2553 scope.go:117] "RemoveContainer" containerID="13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8" Jul 07 20:29:42.118299 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.118268 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:29:42.118400 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.118352 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:29:42.118468 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:42.118412 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:29:43.127916 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:43.127873 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" event={"ID":"00d156cd-6fda-4747-8188-e0913da1e45c","Type":"ContainerStarted","Data":"ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608"} Jul 07 20:29:43.529016 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:43.528964 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:43.529235 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:43.529118 2553 patch_prober.go:28] interesting pod/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 07 20:29:43.529235 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:43.529173 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 07 20:29:44.528391 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:44.528355 2553 patch_prober.go:28] interesting pod/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 07 20:29:44.529018 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:44.528961 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 07 20:29:45.528403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:45.528356 2553 patch_prober.go:28] interesting pod/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 07 20:29:45.528906 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:45.528441 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 07 20:29:46.528350 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:46.527886 2553 patch_prober.go:28] interesting pod/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 07 20:29:46.528350 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:46.527950 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 07 20:29:47.301732 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.301689 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:47.364487 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.364448 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:47.532655 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.532625 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:47.533075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.532967 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:47.533566 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.533549 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:47.609689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.609572 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4"] Jul 07 20:29:47.632885 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.632850 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:47.891658 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:47.891575 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:48.007164 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:48.007122 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:48.103219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:48.103179 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:48.159157 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:48.159086 2553 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" secret="" err="secret \"e2e-mt-e8652739-openshift-default-dockercfg-49v6k\" not found" Jul 07 20:29:48.243070 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:48.243037 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:49.170002 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:49.169941 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" containerID="cri-o://ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608" gracePeriod=30 Jul 07 20:29:54.158206 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.158172 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:54.258482 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.258445 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:54.358871 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.358835 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:54.426092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.426070 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:54.507904 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.507869 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-envoy\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.507944 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-credential-socket\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.507966 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-socket\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.507984 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhbbt\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-kube-api-access-xhbbt\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508029 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d156cd-6fda-4747-8188-e0913da1e45c-istiod-ca-cert\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508055 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-istio-token\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508101 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d156cd-6fda-4747-8188-e0913da1e45c-istio-podinfo\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508127 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-data\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508161 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-certs\") pod \"00d156cd-6fda-4747-8188-e0913da1e45c\" (UID: \"00d156cd-6fda-4747-8188-e0913da1e45c\") " Jul 07 20:29:54.508438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508255 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:29:54.508438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508313 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:29:54.508668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508475 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.508668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508492 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.508668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508517 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00d156cd-6fda-4747-8188-e0913da1e45c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:29:54.508668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508549 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:29:54.508977 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.508957 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-data" (OuterVolumeSpecName: "istio-data") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:29:54.510318 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.510295 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-istio-token" (OuterVolumeSpecName: "istio-token") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:29:54.510421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.510352 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-kube-api-access-xhbbt" (OuterVolumeSpecName: "kube-api-access-xhbbt") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "kube-api-access-xhbbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:29:54.510596 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.510567 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:29:54.510704 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.510632 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00d156cd-6fda-4747-8188-e0913da1e45c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "00d156cd-6fda-4747-8188-e0913da1e45c" (UID: "00d156cd-6fda-4747-8188-e0913da1e45c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:29:54.548506 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.548455 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:29:54.609221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.609188 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhbbt\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-kube-api-access-xhbbt\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.609221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.609215 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d156cd-6fda-4747-8188-e0913da1e45c-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.609221 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.609228 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d156cd-6fda-4747-8188-e0913da1e45c-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.609447 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.609237 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d156cd-6fda-4747-8188-e0913da1e45c-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.609447 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.609245 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.609447 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.609253 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:54.609447 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:54.609260 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d156cd-6fda-4747-8188-e0913da1e45c-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:29:55.203780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.203743 2553 generic.go:356] "Generic (PLEG): container finished" podID="00d156cd-6fda-4747-8188-e0913da1e45c" containerID="ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608" exitCode=0 Jul 07 20:29:55.204259 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.203803 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" Jul 07 20:29:55.204259 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.203804 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" event={"ID":"00d156cd-6fda-4747-8188-e0913da1e45c","Type":"ContainerDied","Data":"ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608"} Jul 07 20:29:55.204259 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.203961 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4" event={"ID":"00d156cd-6fda-4747-8188-e0913da1e45c","Type":"ContainerDied","Data":"5dc5f4270028d510f925ce218765aab7e3b602af53af0b2c3ab935f6ce8fad87"} Jul 07 20:29:55.204259 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.204012 2553 scope.go:117] "RemoveContainer" containerID="ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608" Jul 07 20:29:55.216639 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.216619 2553 scope.go:117] "RemoveContainer" containerID="13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8" Jul 07 20:29:55.224918 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.224900 2553 scope.go:117] "RemoveContainer" containerID="ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608" Jul 07 20:29:55.225178 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:29:55.225159 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608\": container with ID starting with ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608 not found: ID does not exist" containerID="ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608" Jul 07 20:29:55.225241 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.225188 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608"} err="failed to get container status \"ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608\": rpc error: code = NotFound desc = could not find container \"ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608\": container with ID starting with ccc6f46be35d7f2a5bd64df15f0f70934db43e83a1699c6fd2484e280542b608 not found: ID does not exist" Jul 07 20:29:55.225241 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.225205 2553 scope.go:117] "RemoveContainer" containerID="13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8" Jul 07 20:29:55.225419 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:29:55.225405 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8\": container with ID starting with 13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8 not found: ID does not exist" containerID="13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8" Jul 07 20:29:55.225460 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.225423 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8"} err="failed to get container status \"13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8\": rpc error: code = NotFound desc = could not find container \"13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8\": container with ID starting with 13095cb4a0cf19a25d2b47a1562dea0f14cffe0bc54ea17db69c4a2faf9209e8 not found: ID does not exist" Jul 07 20:29:55.230781 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.230757 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4"] Jul 07 20:29:55.234619 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:55.234597 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e8652739-openshift-default-68559bf9dc-n8qh4"] Jul 07 20:29:56.029491 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:29:56.029460 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" path="/var/lib/kubelet/pods/00d156cd-6fda-4747-8188-e0913da1e45c/volumes" Jul 07 20:30:00.565167 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:00.565131 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:00.665105 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:00.665061 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:07.568142 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.568104 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:07.866444 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.866346 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662"] Jul 07 20:30:07.866764 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.866745 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" Jul 07 20:30:07.866764 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.866762 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" Jul 07 20:30:07.866922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.866775 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" Jul 07 20:30:07.866922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.866783 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" Jul 07 20:30:07.866922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.866872 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" Jul 07 20:30:07.866922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.866880 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="00d156cd-6fda-4747-8188-e0913da1e45c" containerName="istio-proxy" Jul 07 20:30:07.871317 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.871298 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.874060 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.873979 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a3bfa138-openshift-default-dockercfg-jp7dp\"" Jul 07 20:30:07.883614 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.883588 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662"] Jul 07 20:30:07.895203 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.895162 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:07.932689 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.932648 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-socket\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.932869 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.932687 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-certs\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.932869 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.932837 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-envoy\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.932954 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.932887 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-podinfo\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.932954 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.932920 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-data\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.932954 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.932948 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-credential-socket\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.933084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.933025 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-token\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.933084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.933057 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prfrn\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-kube-api-access-prfrn\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:07.933159 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:07.933143 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istiod-ca-cert\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036395 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-envoy\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036645 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036605 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-podinfo\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036647 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-data\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036674 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-credential-socket\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036700 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-token\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036872 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036722 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prfrn\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-kube-api-access-prfrn\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036872 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036794 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istiod-ca-cert\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.036872 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036852 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-socket\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.037101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.036874 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-certs\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.037101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.037061 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-credential-socket\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.037387 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.037360 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-data\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.037514 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.037493 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-certs\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.037762 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.037739 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istiod-ca-cert\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.037833 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.037745 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-socket\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.039199 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.039176 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-envoy\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.039282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.039203 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-podinfo\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.046474 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.046443 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-token\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.046742 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.046721 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prfrn\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-kube-api-access-prfrn\") pod \"e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.183755 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.183659 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:08.336186 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.336154 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662"] Jul 07 20:30:08.338888 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:30:08.338856 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b6d58e_ebe8_48b5_8607_916f8eb48e60.slice/crio-65af2f1154a4835630dda3f5c31b8c65fe8dcd763c7e5dd9e98f5076bed9485f WatchSource:0}: Error finding container 65af2f1154a4835630dda3f5c31b8c65fe8dcd763c7e5dd9e98f5076bed9485f: Status 404 returned error can't find the container with id 65af2f1154a4835630dda3f5c31b8c65fe8dcd763c7e5dd9e98f5076bed9485f Jul 07 20:30:08.341030 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.340969 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:30:08.341118 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.341073 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:30:08.341118 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.341105 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:30:08.410848 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:08.410809 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:09.283968 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:09.283928 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" event={"ID":"f1b6d58e-ebe8-48b5-8607-916f8eb48e60","Type":"ContainerStarted","Data":"0525484706c0945473890d8fa6800cf88c4e957a83f3dcea0da7bbbb7e3cd183"} Jul 07 20:30:09.284428 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:09.283985 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" event={"ID":"f1b6d58e-ebe8-48b5-8607-916f8eb48e60","Type":"ContainerStarted","Data":"65af2f1154a4835630dda3f5c31b8c65fe8dcd763c7e5dd9e98f5076bed9485f"} Jul 07 20:30:09.308030 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:09.307960 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" podStartSLOduration=2.307939322 podStartE2EDuration="2.307939322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:30:09.305925418 +0000 UTC m=+1761.814597992" watchObservedRunningTime="2026-07-07 20:30:09.307939322 +0000 UTC m=+1761.816611897" Jul 07 20:30:10.184628 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:10.184598 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:10.189787 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:10.189761 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:10.290003 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:10.289944 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:10.291051 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:10.291022 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:11.292310 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:11.292273 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:11.371849 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:11.371785 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:11.441902 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:11.441866 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:11.739771 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:11.739728 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:11.956927 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:11.956890 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:12.056959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:12.056857 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:15.801714 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:15.801671 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:17.746894 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:17.746858 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:17.846815 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:17.846762 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:24.375449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:24.375412 2553 generic.go:356] "Generic (PLEG): container finished" podID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerID="0525484706c0945473890d8fa6800cf88c4e957a83f3dcea0da7bbbb7e3cd183" exitCode=0 Jul 07 20:30:24.375883 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:24.375490 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" event={"ID":"f1b6d58e-ebe8-48b5-8607-916f8eb48e60","Type":"ContainerDied","Data":"0525484706c0945473890d8fa6800cf88c4e957a83f3dcea0da7bbbb7e3cd183"} Jul 07 20:30:24.375933 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:24.375911 2553 scope.go:117] "RemoveContainer" containerID="0525484706c0945473890d8fa6800cf88c4e957a83f3dcea0da7bbbb7e3cd183" Jul 07 20:30:24.376969 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:24.376943 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:30:24.377067 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:24.377049 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:30:24.377107 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:24.377090 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:30:25.290287 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:25.290245 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:25.384022 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:25.383965 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" event={"ID":"f1b6d58e-ebe8-48b5-8607-916f8eb48e60","Type":"ContainerStarted","Data":"dddb1bcac7a1f35cd1b081ba3eaa5d7ac66d5a443c1c213664ffb0b0617f7d51"} Jul 07 20:30:26.184520 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:26.184492 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:26.185721 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:26.185694 2553 patch_prober.go:28] interesting pod/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 07 20:30:26.185860 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:26.185748 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 07 20:30:27.184801 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:27.184761 2553 patch_prober.go:28] interesting pod/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 07 20:30:27.185324 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:27.184826 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 07 20:30:28.184930 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:28.184892 2553 patch_prober.go:28] interesting pod/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 07 20:30:28.185352 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:28.184951 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 07 20:30:28.734730 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:28.734683 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:29.184346 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:29.184250 2553 patch_prober.go:28] interesting pod/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 07 20:30:29.184346 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:29.184309 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 07 20:30:30.188951 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:30.188918 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:30.189359 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:30.189338 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:30.190014 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:30.189976 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:30:33.901008 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:33.900957 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:34.000805 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:34.000766 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:48.098604 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:48.098570 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:30:48.103578 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:48.103551 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:30:48.345383 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:48.345344 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:48.719861 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:48.719822 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:54.502075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:54.502036 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:30:59.586787 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:30:59.586749 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:35.976868 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:35.976819 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:37.293780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:37.293724 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:44.256237 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:44.256193 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:44.352142 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:44.352101 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:44.577191 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:44.577100 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662"] Jul 07 20:31:44.577432 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:44.577396 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" containerID="cri-o://dddb1bcac7a1f35cd1b081ba3eaa5d7ac66d5a443c1c213664ffb0b0617f7d51" gracePeriod=30 Jul 07 20:31:44.587431 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:44.587397 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:44.880848 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:44.880711 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:45.192219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:45.192129 2553 patch_prober.go:28] interesting pod/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 20:31:45.192219 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:45.192206 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 20:31:46.066174 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:46.066128 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:46.313258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:46.313220 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:46.406025 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:46.405899 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:47.153186 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:47.153138 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:47.289142 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:47.289093 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:47.422710 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:47.422621 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:48.932865 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:48.932824 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:49.849679 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:49.849599 2553 generic.go:356] "Generic (PLEG): container finished" podID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerID="dddb1bcac7a1f35cd1b081ba3eaa5d7ac66d5a443c1c213664ffb0b0617f7d51" exitCode=0 Jul 07 20:31:49.849679 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:49.849668 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" event={"ID":"f1b6d58e-ebe8-48b5-8607-916f8eb48e60","Type":"ContainerDied","Data":"dddb1bcac7a1f35cd1b081ba3eaa5d7ac66d5a443c1c213664ffb0b0617f7d51"} Jul 07 20:31:49.849866 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:49.849752 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" event={"ID":"f1b6d58e-ebe8-48b5-8607-916f8eb48e60","Type":"ContainerDied","Data":"65af2f1154a4835630dda3f5c31b8c65fe8dcd763c7e5dd9e98f5076bed9485f"} Jul 07 20:31:49.849866 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:49.849771 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65af2f1154a4835630dda3f5c31b8c65fe8dcd763c7e5dd9e98f5076bed9485f" Jul 07 20:31:49.849866 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:49.849785 2553 scope.go:117] "RemoveContainer" containerID="0525484706c0945473890d8fa6800cf88c4e957a83f3dcea0da7bbbb7e3cd183" Jul 07 20:31:49.855926 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:49.855896 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:31:50.037985 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.037956 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-credential-socket\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038060 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-token\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038088 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istiod-ca-cert\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038155 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-envoy\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038237 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038282 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-socket\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038310 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-podinfo\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038337 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-data\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038359 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prfrn\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-kube-api-access-prfrn\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038464 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038397 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-certs\") pod \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\" (UID: \"f1b6d58e-ebe8-48b5-8607-916f8eb48e60\") " Jul 07 20:31:50.038919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038489 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:31:50.038919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038526 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:31:50.038919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038677 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:31:50.038919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038777 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.038919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038794 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.038919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038808 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.038919 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.038822 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.039292 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.039155 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:31:50.040745 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.040704 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:31:50.040862 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.040842 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-kube-api-access-prfrn" (OuterVolumeSpecName: "kube-api-access-prfrn") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "kube-api-access-prfrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:31:50.041059 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.041036 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:31:50.041222 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.041193 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1b6d58e-ebe8-48b5-8607-916f8eb48e60" (UID: "f1b6d58e-ebe8-48b5-8607-916f8eb48e60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:31:50.139759 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.139677 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.139759 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.139708 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.139759 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.139716 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.139759 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.139725 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.139759 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.139733 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prfrn\" (UniqueName: \"kubernetes.io/projected/f1b6d58e-ebe8-48b5-8607-916f8eb48e60-kube-api-access-prfrn\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:31:50.856535 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.856504 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662" Jul 07 20:31:50.896023 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.895969 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662"] Jul 07 20:31:50.902098 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:50.902066 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a3bfa138-openshift-default-7f9b87c647-5w662"] Jul 07 20:31:52.030609 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:52.030574 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" path="/var/lib/kubelet/pods/f1b6d58e-ebe8-48b5-8607-916f8eb48e60/volumes" Jul 07 20:31:55.572814 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:55.572767 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:55.673092 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:55.673037 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:31:56.319795 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:31:56.319762 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:02.789542 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:02.789486 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:02.842956 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:02.842912 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:02.931410 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:02.931364 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:03.024178 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:03.024142 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:03.187964 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:03.187867 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:03.411611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:03.411573 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:13.554504 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:13.554459 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:13.928736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:13.928639 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:34.930805 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:34.930768 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:40.899604 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:40.899561 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:40.958348 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:40.958305 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:41.257888 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.257854 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:41.598013 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.597911 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l"] Jul 07 20:32:41.598333 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.598320 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" Jul 07 20:32:41.598379 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.598336 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" Jul 07 20:32:41.598379 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.598349 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" Jul 07 20:32:41.598379 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.598355 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" Jul 07 20:32:41.598478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.598429 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" Jul 07 20:32:41.598478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.598436 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1b6d58e-ebe8-48b5-8607-916f8eb48e60" containerName="istio-proxy" Jul 07 20:32:41.601563 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.601543 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.604407 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.604387 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c6e6f8-openshift-default-dockercfg-mm52v\"" Jul 07 20:32:41.613746 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.613717 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l"] Jul 07 20:32:41.617538 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.617507 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:41.715017 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.714941 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-socket\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715209 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715096 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-podinfo\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715209 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715173 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxv84\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-kube-api-access-rxv84\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715354 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715278 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b69154-a881-4d41-bde6-0bfa6ae16559-istiod-ca-cert\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715393 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715355 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-envoy\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715520 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715505 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-token\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715573 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715544 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-credential-socket\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715573 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715567 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-certs\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.715678 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.715614 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-data\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817288 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817250 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-data\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817311 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-socket\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817347 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-podinfo\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817363 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxv84\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-kube-api-access-rxv84\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817400 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b69154-a881-4d41-bde6-0bfa6ae16559-istiod-ca-cert\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817440 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-envoy\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817478 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817469 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-token\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817802 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817512 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-credential-socket\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817802 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817542 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-certs\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817802 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817774 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-data\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817965 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817783 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-socket\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.817965 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817883 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-certs\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.818110 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.817978 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-credential-socket\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.818248 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.818224 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b69154-a881-4d41-bde6-0bfa6ae16559-istiod-ca-cert\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.819846 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.819825 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-envoy\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.819973 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.819954 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-podinfo\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.825697 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.825670 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-token\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.825859 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.825844 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxv84\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-kube-api-access-rxv84\") pod \"e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:41.837609 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.837561 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:41.913849 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:41.913758 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:42.063047 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:42.063018 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l"] Jul 07 20:32:42.065053 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:32:42.065024 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b69154_a881_4d41_bde6_0bfa6ae16559.slice/crio-f4eb693479870365cf58098516ee4f57736a2747137fc353580f5077aa63f25a WatchSource:0}: Error finding container f4eb693479870365cf58098516ee4f57736a2747137fc353580f5077aa63f25a: Status 404 returned error can't find the container with id f4eb693479870365cf58098516ee4f57736a2747137fc353580f5077aa63f25a Jul 07 20:32:42.067213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:42.067176 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:42.067331 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:42.067242 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:42.067331 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:42.067271 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:42.135435 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:42.135398 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" event={"ID":"82b69154-a881-4d41-bde6-0bfa6ae16559","Type":"ContainerStarted","Data":"f4eb693479870365cf58098516ee4f57736a2747137fc353580f5077aa63f25a"} Jul 07 20:32:43.142523 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:43.142484 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" event={"ID":"82b69154-a881-4d41-bde6-0bfa6ae16559","Type":"ContainerStarted","Data":"288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86"} Jul 07 20:32:43.170304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:43.170251 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" podStartSLOduration=2.170233023 podStartE2EDuration="2.170233023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:32:43.16809332 +0000 UTC m=+1915.676765896" watchObservedRunningTime="2026-07-07 20:32:43.170233023 +0000 UTC m=+1915.678905598" Jul 07 20:32:43.914386 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:43.914346 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:43.919390 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:43.919360 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:44.148233 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:44.148202 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:44.149168 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:44.149150 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:45.306697 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:45.306602 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:45.403665 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:45.403598 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:45.547138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:45.547101 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:45.676260 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:45.676158 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:46.267826 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:46.267788 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:46.368052 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:46.367975 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:49.764678 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:49.764643 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:50.135323 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.135220 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:50.423430 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.423335 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv"] Jul 07 20:32:50.428120 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.428091 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.430934 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.430909 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-080fc2-openshift-default-dockercfg-xssgr\"" Jul 07 20:32:50.439026 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.438978 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv"] Jul 07 20:32:50.450134 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.450099 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:50.504545 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504506 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec0c21c-4b61-4756-88eb-32912efe54b2-istiod-ca-cert\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504563 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-certs\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504628 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-token\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504722 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-podinfo\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504851 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504795 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-envoy\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504851 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504840 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-socket\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504924 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504857 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-data\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504924 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504876 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt98q\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-kube-api-access-jt98q\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.504924 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.504900 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-credential-socket\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.547583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.547546 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:50.605824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.605767 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-socket\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.605834 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-data\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.605869 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt98q\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-kube-api-access-jt98q\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.605898 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-credential-socket\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.605966 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec0c21c-4b61-4756-88eb-32912efe54b2-istiod-ca-cert\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.606021 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-certs\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606333 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.606083 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-token\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606333 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.606128 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-podinfo\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606333 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.606198 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-envoy\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.606333 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.606272 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-socket\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.607502 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.607014 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec0c21c-4b61-4756-88eb-32912efe54b2-istiod-ca-cert\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.607502 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.607174 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-data\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.607502 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.607461 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-certs\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.607747 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.607575 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-credential-socket\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.612013 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.609103 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-envoy\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.612013 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.610748 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-podinfo\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.619840 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.619755 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-token\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.619840 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.619800 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt98q\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-kube-api-access-jt98q\") pod \"e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.740164 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.740129 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:50.934038 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.934004 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv"] Jul 07 20:32:50.936240 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:32:50.936213 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec0c21c_4b61_4756_88eb_32912efe54b2.slice/crio-f161223ce8219799bdefa11352cbb2d30063cd74935517f121597e3454ead258 WatchSource:0}: Error finding container f161223ce8219799bdefa11352cbb2d30063cd74935517f121597e3454ead258: Status 404 returned error can't find the container with id f161223ce8219799bdefa11352cbb2d30063cd74935517f121597e3454ead258 Jul 07 20:32:50.938965 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.938931 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:50.939047 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.939034 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:50.939089 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:50.939074 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:51.189111 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:51.189069 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" event={"ID":"1ec0c21c-4b61-4756-88eb-32912efe54b2","Type":"ContainerStarted","Data":"6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f"} Jul 07 20:32:51.189288 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:51.189174 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" event={"ID":"1ec0c21c-4b61-4756-88eb-32912efe54b2","Type":"ContainerStarted","Data":"f161223ce8219799bdefa11352cbb2d30063cd74935517f121597e3454ead258"} Jul 07 20:32:51.211560 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:51.211482 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podStartSLOduration=1.211444235 podStartE2EDuration="1.211444235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:32:51.209101393 +0000 UTC m=+1923.717773968" watchObservedRunningTime="2026-07-07 20:32:51.211444235 +0000 UTC m=+1923.720116810" Jul 07 20:32:51.741142 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:51.741104 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:52.746320 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:52.746289 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:53.170938 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.170840 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:53.200834 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.200799 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:53.201862 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.201841 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:32:53.235923 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.235888 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:53.332038 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.331980 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:53.526720 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.526676 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:53.753603 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.753566 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:53.855149 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:53.855059 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:57.746716 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:57.746670 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:32:58.231594 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:58.231565 2553 generic.go:356] "Generic (PLEG): container finished" podID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerID="288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86" exitCode=0 Jul 07 20:32:58.231763 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:58.231640 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" event={"ID":"82b69154-a881-4d41-bde6-0bfa6ae16559","Type":"ContainerDied","Data":"288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86"} Jul 07 20:32:58.232095 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:58.232080 2553 scope.go:117] "RemoveContainer" containerID="288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86" Jul 07 20:32:58.233113 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:58.233088 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:58.233167 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:58.233157 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:58.233207 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:58.233189 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:32:59.148624 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:59.148585 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:59.239684 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:59.239643 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" event={"ID":"82b69154-a881-4d41-bde6-0bfa6ae16559","Type":"ContainerStarted","Data":"e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6"} Jul 07 20:32:59.914496 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:59.914452 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:32:59.916089 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:59.916059 2553 patch_prober.go:28] interesting pod/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 07 20:32:59.916306 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:32:59.916276 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 07 20:33:00.914624 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:00.914578 2553 patch_prober.go:28] interesting pod/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 07 20:33:00.915282 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:00.915247 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 07 20:33:01.914509 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:01.914465 2553 patch_prober.go:28] interesting pod/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 07 20:33:01.914830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:01.914801 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 07 20:33:02.914605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:02.914555 2553 patch_prober.go:28] interesting pod/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 07 20:33:02.914839 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:02.914627 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 07 20:33:03.824051 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:03.824009 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:33:03.921575 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:03.921539 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:33:03.923549 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:03.923524 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:33:03.926850 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:03.926831 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:33:06.841234 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:06.841190 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:33:06.941748 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:06.941700 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:33:09.036116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.034376 2553 patch_prober.go:28] interesting pod/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:51616->10.132.0.63:15021: read: connection reset by peer" start-of-body= Jul 07 20:33:09.036116 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.034441 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:51616->10.132.0.63:15021: read: connection reset by peer" Jul 07 20:33:09.080789 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.080752 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:33:09.305262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.305178 2553 generic.go:356] "Generic (PLEG): container finished" podID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerID="6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f" exitCode=0 Jul 07 20:33:09.305262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.305253 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" event={"ID":"1ec0c21c-4b61-4756-88eb-32912efe54b2","Type":"ContainerDied","Data":"6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f"} Jul 07 20:33:09.305711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.305691 2553 scope.go:117] "RemoveContainer" containerID="6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f" Jul 07 20:33:09.306878 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.306847 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:33:09.306947 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.306933 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:33:09.307012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:09.306979 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:33:10.314455 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:10.314418 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" event={"ID":"1ec0c21c-4b61-4756-88eb-32912efe54b2","Type":"ContainerStarted","Data":"752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967"} Jul 07 20:33:10.740825 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:10.740790 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:33:10.741029 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:10.740985 2553 patch_prober.go:28] interesting pod/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 07 20:33:10.741080 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:10.741061 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 07 20:33:11.740685 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:11.740645 2553 patch_prober.go:28] interesting pod/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 07 20:33:11.741114 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:11.740716 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 07 20:33:12.740908 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:12.740867 2553 patch_prober.go:28] interesting pod/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 07 20:33:12.741342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:12.740926 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 07 20:33:13.740939 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:13.740899 2553 patch_prober.go:28] interesting pod/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 07 20:33:13.741389 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:13.740973 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 07 20:33:14.740808 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:14.740772 2553 patch_prober.go:28] interesting pod/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 07 20:33:14.741045 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:14.740831 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 07 20:33:15.450842 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:15.450808 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:33:15.551205 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:15.551167 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:33:15.744504 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:15.744477 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:33:15.744913 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:15.744838 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:33:15.745647 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:15.745626 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:33:45.806787 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:33:45.806690 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:11.995841 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:11.995797 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:14.621657 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:14.621615 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:18.628033 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:18.627983 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:35.649768 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:35.649724 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:55.721222 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:55.721183 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:55.987733 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:55.987693 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l"] Jul 07 20:34:55.988183 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:55.987954 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" containerID="cri-o://e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6" gracePeriod=30 Jul 07 20:34:56.014592 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:56.014556 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:56.281140 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:56.281053 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:56.459599 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:56.459563 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:56.560084 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:56.559978 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:56.662842 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:56.662804 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:56.893728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:56.893643 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:57.072749 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:57.072714 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:57.171878 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:57.171788 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:34:57.324480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:34:57.324441 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:01.254300 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.254237 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:35:01.306112 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306070 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-token\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306127 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-data\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306178 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-credential-socket\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306205 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-certs\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306250 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-socket\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306527 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306306 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b69154-a881-4d41-bde6-0bfa6ae16559-istiod-ca-cert\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306527 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306384 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-podinfo\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306527 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306412 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxv84\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-kube-api-access-rxv84\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306527 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306444 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-envoy\") pod \"82b69154-a881-4d41-bde6-0bfa6ae16559\" (UID: \"82b69154-a881-4d41-bde6-0bfa6ae16559\") " Jul 07 20:35:01.306736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306525 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:01.306736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306543 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:01.306835 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306813 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.306891 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306834 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.307075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.306972 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-data" (OuterVolumeSpecName: "istio-data") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:01.307075 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.307035 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:01.307229 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.307125 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b69154-a881-4d41-bde6-0bfa6ae16559-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:35:01.308519 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.308495 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-token" (OuterVolumeSpecName: "istio-token") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:35:01.308670 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.308650 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:35:01.308738 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.308662 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-kube-api-access-rxv84" (OuterVolumeSpecName: "kube-api-access-rxv84") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "kube-api-access-rxv84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:35:01.309068 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.309048 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82b69154-a881-4d41-bde6-0bfa6ae16559" (UID: "82b69154-a881-4d41-bde6-0bfa6ae16559"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:01.407331 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.407285 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b69154-a881-4d41-bde6-0bfa6ae16559-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.407331 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.407320 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.407331 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.407334 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxv84\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-kube-api-access-rxv84\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.407583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.407348 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.407583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.407360 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.407583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.407369 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.407583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.407380 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b69154-a881-4d41-bde6-0bfa6ae16559-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:01.983046 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.982984 2553 generic.go:356] "Generic (PLEG): container finished" podID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerID="e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6" exitCode=0 Jul 07 20:35:01.983228 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.983059 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" Jul 07 20:35:01.983228 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.983097 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" event={"ID":"82b69154-a881-4d41-bde6-0bfa6ae16559","Type":"ContainerDied","Data":"e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6"} Jul 07 20:35:01.983228 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.983157 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l" event={"ID":"82b69154-a881-4d41-bde6-0bfa6ae16559","Type":"ContainerDied","Data":"f4eb693479870365cf58098516ee4f57736a2747137fc353580f5077aa63f25a"} Jul 07 20:35:01.983228 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.983188 2553 scope.go:117] "RemoveContainer" containerID="e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6" Jul 07 20:35:01.993293 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:01.993268 2553 scope.go:117] "RemoveContainer" containerID="288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86" Jul 07 20:35:02.001980 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:02.001958 2553 scope.go:117] "RemoveContainer" containerID="e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6" Jul 07 20:35:02.002351 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:35:02.002329 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6\": container with ID starting with e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6 not found: ID does not exist" containerID="e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6" Jul 07 20:35:02.002397 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:02.002362 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6"} err="failed to get container status \"e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6\": rpc error: code = NotFound desc = could not find container \"e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6\": container with ID starting with e24a749f802530d3d0c71f290e8b5c4688e594c471cbcb3f53f65d4c5e15d9e6 not found: ID does not exist" Jul 07 20:35:02.002397 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:02.002380 2553 scope.go:117] "RemoveContainer" containerID="288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86" Jul 07 20:35:02.002650 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:35:02.002630 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86\": container with ID starting with 288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86 not found: ID does not exist" containerID="288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86" Jul 07 20:35:02.002712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:02.002655 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86"} err="failed to get container status \"288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86\": rpc error: code = NotFound desc = could not find container \"288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86\": container with ID starting with 288968f0a226a69ba739af8050fb376e02a36219b7cd9012f548821b24928e86 not found: ID does not exist" Jul 07 20:35:02.030003 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:02.029963 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l"] Jul 07 20:35:02.030162 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:02.030016 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c6e6f8-openshift-default-744f779849-nhq6l"] Jul 07 20:35:03.144678 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:03.144633 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:03.213971 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:03.213935 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:03.390237 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:03.390194 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv"] Jul 07 20:35:03.390487 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:03.390462 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" containerID="cri-o://752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967" gracePeriod=30 Jul 07 20:35:03.408055 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:03.407946 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:03.860859 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:03.860811 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:04.029703 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:04.029671 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" path="/var/lib/kubelet/pods/82b69154-a881-4d41-bde6-0bfa6ae16559/volumes" Jul 07 20:35:04.047897 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:04.047864 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:04.374425 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:04.374385 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:04.525038 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:04.525001 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:08.665737 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.665663 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:35:08.786074 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786030 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-envoy\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786074 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786075 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-credential-socket\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786296 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786102 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-certs\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786296 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786153 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-token\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786296 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786182 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-podinfo\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786296 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786196 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-socket\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786296 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786217 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt98q\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-kube-api-access-jt98q\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786296 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786276 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-data\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786606 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786314 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec0c21c-4b61-4756-88eb-32912efe54b2-istiod-ca-cert\") pod \"1ec0c21c-4b61-4756-88eb-32912efe54b2\" (UID: \"1ec0c21c-4b61-4756-88eb-32912efe54b2\") " Jul 07 20:35:08.786606 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786459 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:08.786606 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786450 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:08.786606 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786535 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:08.786832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786698 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.786832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786718 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.786832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786734 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.786832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.786763 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ec0c21c-4b61-4756-88eb-32912efe54b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:35:08.787243 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.787218 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:08.788528 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.788491 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-kube-api-access-jt98q" (OuterVolumeSpecName: "kube-api-access-jt98q") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "kube-api-access-jt98q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:35:08.788652 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.788562 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:35:08.788757 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.788736 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:08.788803 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.788739 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ec0c21c-4b61-4756-88eb-32912efe54b2" (UID: "1ec0c21c-4b61-4756-88eb-32912efe54b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:35:08.887531 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.887488 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.887531 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.887519 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.887531 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.887529 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.887531 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.887539 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt98q\" (UniqueName: \"kubernetes.io/projected/1ec0c21c-4b61-4756-88eb-32912efe54b2-kube-api-access-jt98q\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.887806 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.887548 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec0c21c-4b61-4756-88eb-32912efe54b2-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:08.887806 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:08.887556 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec0c21c-4b61-4756-88eb-32912efe54b2-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:09.024080 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.024049 2553 generic.go:356] "Generic (PLEG): container finished" podID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerID="752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967" exitCode=0 Jul 07 20:35:09.024252 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.024103 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" Jul 07 20:35:09.024252 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.024103 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" event={"ID":"1ec0c21c-4b61-4756-88eb-32912efe54b2","Type":"ContainerDied","Data":"752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967"} Jul 07 20:35:09.024252 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.024219 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv" event={"ID":"1ec0c21c-4b61-4756-88eb-32912efe54b2","Type":"ContainerDied","Data":"f161223ce8219799bdefa11352cbb2d30063cd74935517f121597e3454ead258"} Jul 07 20:35:09.024252 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.024249 2553 scope.go:117] "RemoveContainer" containerID="752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967" Jul 07 20:35:09.034560 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.034542 2553 scope.go:117] "RemoveContainer" containerID="6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f" Jul 07 20:35:09.043643 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.043619 2553 scope.go:117] "RemoveContainer" containerID="752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967" Jul 07 20:35:09.043951 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:35:09.043911 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967\": container with ID starting with 752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967 not found: ID does not exist" containerID="752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967" Jul 07 20:35:09.044048 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.043961 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967"} err="failed to get container status \"752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967\": rpc error: code = NotFound desc = could not find container \"752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967\": container with ID starting with 752c822f105233d2dd99ff10f1c1f94419a31a99d261038dc4009cff41c98967 not found: ID does not exist" Jul 07 20:35:09.044048 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.043986 2553 scope.go:117] "RemoveContainer" containerID="6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f" Jul 07 20:35:09.044244 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:35:09.044224 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f\": container with ID starting with 6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f not found: ID does not exist" containerID="6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f" Jul 07 20:35:09.044286 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.044250 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f"} err="failed to get container status \"6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f\": rpc error: code = NotFound desc = could not find container \"6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f\": container with ID starting with 6e69f98f9df211294cbf8397dc1099d227ba5ed1d675f437723794f449d9858f not found: ID does not exist" Jul 07 20:35:09.048825 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.048799 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv"] Jul 07 20:35:09.054072 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:09.054047 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-080fc2-openshift-default-7dbf4844cc-5ssgv"] Jul 07 20:35:10.030466 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:10.030430 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" path="/var/lib/kubelet/pods/1ec0c21c-4b61-4756-88eb-32912efe54b2/volumes" Jul 07 20:35:10.307974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:10.307864 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:10.389427 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:10.389338 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:10.461844 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:10.461809 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:10.569162 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:10.569068 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:11.034104 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:11.034069 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:11.135461 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:11.135426 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:14.852333 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:14.852294 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:15.351023 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.350968 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:15.674964 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.674876 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s"] Jul 07 20:35:15.675495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675473 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" Jul 07 20:35:15.675495 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675493 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" Jul 07 20:35:15.675580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675505 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" Jul 07 20:35:15.675580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675514 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" Jul 07 20:35:15.675580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675526 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" Jul 07 20:35:15.675580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675536 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" Jul 07 20:35:15.675580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675546 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" Jul 07 20:35:15.675580 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675556 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" Jul 07 20:35:15.675774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675645 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" Jul 07 20:35:15.675774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675656 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" Jul 07 20:35:15.675774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.675668 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ec0c21c-4b61-4756-88eb-32912efe54b2" containerName="istio-proxy" Jul 07 20:35:15.680600 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.680569 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.683635 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.683613 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c9e405-openshift-default-dockercfg-6k68p\"" Jul 07 20:35:15.687645 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.687622 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s"] Jul 07 20:35:15.696777 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.696732 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:15.750604 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750557 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-credential-socket\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750797 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750616 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-socket\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750797 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750652 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-certs\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750797 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750674 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-envoy\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750797 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750708 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf22db-e137-4298-a4da-aefd76781fce-istiod-ca-cert\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750797 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750763 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf22db-e137-4298-a4da-aefd76781fce-istio-podinfo\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750829 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-istio-token\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750853 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8nd5\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-kube-api-access-h8nd5\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.750983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.750871 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-data\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.851817 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.851780 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-istio-token\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.851817 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.851818 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8nd5\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-kube-api-access-h8nd5\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.851840 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-data\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.851888 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-credential-socket\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.851910 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-socket\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.851934 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-certs\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852065 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-envoy\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852133 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf22db-e137-4298-a4da-aefd76781fce-istiod-ca-cert\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852179 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf22db-e137-4298-a4da-aefd76781fce-istio-podinfo\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852337 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-data\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852396 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-credential-socket\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852453 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-certs\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852711 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852474 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-socket\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.852910 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.852821 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf22db-e137-4298-a4da-aefd76781fce-istiod-ca-cert\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.854397 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.854367 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-envoy\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.854622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.854602 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf22db-e137-4298-a4da-aefd76781fce-istio-podinfo\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.860970 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.860941 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-istio-token\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.861091 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.861026 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8nd5\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-kube-api-access-h8nd5\") pod \"e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:15.893368 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.893324 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:15.993417 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:15.993379 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:16.145954 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:16.145927 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s"] Jul 07 20:35:16.148045 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:35:16.148015 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83bf22db_e137_4298_a4da_aefd76781fce.slice/crio-b67641860f7f2d2eaf7272240a7fdf57790a2991d31de16e3569e8fb3e2bfe6d WatchSource:0}: Error finding container b67641860f7f2d2eaf7272240a7fdf57790a2991d31de16e3569e8fb3e2bfe6d: Status 404 returned error can't find the container with id b67641860f7f2d2eaf7272240a7fdf57790a2991d31de16e3569e8fb3e2bfe6d Jul 07 20:35:16.150569 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:16.150533 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:16.150668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:16.150620 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:16.150740 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:16.150665 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:17.074445 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:17.074404 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" event={"ID":"83bf22db-e137-4298-a4da-aefd76781fce","Type":"ContainerStarted","Data":"019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc"} Jul 07 20:35:17.074968 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:17.074473 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" event={"ID":"83bf22db-e137-4298-a4da-aefd76781fce","Type":"ContainerStarted","Data":"b67641860f7f2d2eaf7272240a7fdf57790a2991d31de16e3569e8fb3e2bfe6d"} Jul 07 20:35:17.099933 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:17.099873 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" podStartSLOduration=2.099856343 podStartE2EDuration="2.099856343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:35:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:35:17.097926116 +0000 UTC m=+2069.606598690" watchObservedRunningTime="2026-07-07 20:35:17.099856343 +0000 UTC m=+2069.608528918" Jul 07 20:35:17.994268 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:17.994223 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:17.999385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:17.999356 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:18.080135 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:18.080103 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:18.081077 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:18.081057 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:18.562044 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:18.562002 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:18.670769 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:18.670738 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:19.105370 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:19.105326 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:19.205640 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:19.205597 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:19.367758 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:19.367666 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:19.898974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:19.898936 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:19.998610 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:19.998571 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:20.098418 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:20.098379 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:23.668237 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:23.668198 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:31.777230 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:31.777196 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:31.877644 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:31.877606 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:32.050867 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:32.050762 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:32.117909 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:32.117827 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:32.480605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:32.480561 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:34.122218 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:34.122167 2553 patch_prober.go:28] interesting pod/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 20:35:34.122635 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:34.122252 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 20:35:36.184404 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:36.184374 2553 generic.go:356] "Generic (PLEG): container finished" podID="83bf22db-e137-4298-a4da-aefd76781fce" containerID="019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc" exitCode=0 Jul 07 20:35:36.184832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:36.184433 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" event={"ID":"83bf22db-e137-4298-a4da-aefd76781fce","Type":"ContainerDied","Data":"019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc"} Jul 07 20:35:36.184896 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:36.184878 2553 scope.go:117] "RemoveContainer" containerID="019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc" Jul 07 20:35:36.185935 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:36.185903 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:36.186080 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:36.186001 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:36.186080 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:36.186056 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:37.192443 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:37.192406 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" event={"ID":"83bf22db-e137-4298-a4da-aefd76781fce","Type":"ContainerStarted","Data":"01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1"} Jul 07 20:35:37.994022 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:37.993973 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:37.994263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:37.994232 2553 patch_prober.go:28] interesting pod/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 07 20:35:37.994432 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:37.994296 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 07 20:35:38.993812 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:38.993777 2553 patch_prober.go:28] interesting pod/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 07 20:35:38.994226 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:38.993833 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 07 20:35:38.998729 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:38.998696 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:39.097797 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:39.097754 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:39.328284 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:39.328166 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:39.529406 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:39.529364 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:39.727081 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:39.726789 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:39.994052 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:39.993973 2553 patch_prober.go:28] interesting pod/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 07 20:35:39.994462 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:39.994086 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 07 20:35:40.187388 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:40.187350 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:40.292258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:40.292154 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:40.389273 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:40.389234 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:40.546743 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:40.546628 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:40.994708 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:40.994675 2553 patch_prober.go:28] interesting pod/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 07 20:35:40.995144 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:40.994745 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 07 20:35:41.997624 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:41.997588 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:41.998093 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:41.997940 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:41.998702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:41.998683 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:46.404125 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:46.404069 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:46.463821 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:46.463783 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:46.697238 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:46.697153 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s"] Jul 07 20:35:46.697435 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:46.697410 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" containerID="cri-o://01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1" gracePeriod=30 Jul 07 20:35:46.722503 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:46.722459 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:46.964117 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:46.964021 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:47.343466 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:47.343426 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:47.665398 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:47.665310 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:48.078853 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:48.078817 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:48.140128 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:48.140101 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:35:48.146764 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:48.146735 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:35:50.508750 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:50.508698 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:50.591296 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:50.591258 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:50.659739 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:50.659698 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:50.802957 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:50.802861 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:51.128935 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.128844 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:51.897231 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.897162 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:51.959859 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.959788 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:51.996266 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996234 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-socket\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996467 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996311 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-istio-token\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996467 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996353 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8nd5\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-kube-api-access-h8nd5\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996467 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996383 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf22db-e137-4298-a4da-aefd76781fce-istiod-ca-cert\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996641 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996506 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-envoy\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996641 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996560 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-certs\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996641 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996619 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-data\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996812 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996673 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-credential-socket\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996812 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996729 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf22db-e137-4298-a4da-aefd76781fce-istio-podinfo\") pod \"83bf22db-e137-4298-a4da-aefd76781fce\" (UID: \"83bf22db-e137-4298-a4da-aefd76781fce\") " Jul 07 20:35:51.996812 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996593 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:51.996974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996830 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:51.996974 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.996854 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bf22db-e137-4298-a4da-aefd76781fce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:35:51.997160 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.997139 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83bf22db-e137-4298-a4da-aefd76781fce-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:51.997327 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.997306 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:51.997415 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.997330 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:51.997610 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.997571 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-data" (OuterVolumeSpecName: "istio-data") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:51.997610 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.997589 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:51.998662 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.998642 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-istio-token" (OuterVolumeSpecName: "istio-token") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:35:51.998959 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.998941 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-kube-api-access-h8nd5" (OuterVolumeSpecName: "kube-api-access-h8nd5") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "kube-api-access-h8nd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:35:51.999152 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.999115 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83bf22db-e137-4298-a4da-aefd76781fce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:35:51.999152 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:51.999129 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83bf22db-e137-4298-a4da-aefd76781fce" (UID: "83bf22db-e137-4298-a4da-aefd76781fce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:35:52.098753 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.098711 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8nd5\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-kube-api-access-h8nd5\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:52.098753 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.098745 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:52.098753 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.098757 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:52.099050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.098769 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83bf22db-e137-4298-a4da-aefd76781fce-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:52.099050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.098781 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83bf22db-e137-4298-a4da-aefd76781fce-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:52.099050 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.098792 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83bf22db-e137-4298-a4da-aefd76781fce-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:35:52.300439 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.300403 2553 generic.go:356] "Generic (PLEG): container finished" podID="83bf22db-e137-4298-a4da-aefd76781fce" containerID="01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1" exitCode=0 Jul 07 20:35:52.300591 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.300463 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" Jul 07 20:35:52.300591 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.300489 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" event={"ID":"83bf22db-e137-4298-a4da-aefd76781fce","Type":"ContainerDied","Data":"01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1"} Jul 07 20:35:52.300591 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.300558 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s" event={"ID":"83bf22db-e137-4298-a4da-aefd76781fce","Type":"ContainerDied","Data":"b67641860f7f2d2eaf7272240a7fdf57790a2991d31de16e3569e8fb3e2bfe6d"} Jul 07 20:35:52.300591 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.300583 2553 scope.go:117] "RemoveContainer" containerID="01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1" Jul 07 20:35:52.310038 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.310015 2553 scope.go:117] "RemoveContainer" containerID="019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc" Jul 07 20:35:52.318421 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.318402 2553 scope.go:117] "RemoveContainer" containerID="01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1" Jul 07 20:35:52.318709 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:35:52.318685 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1\": container with ID starting with 01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1 not found: ID does not exist" containerID="01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1" Jul 07 20:35:52.318807 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.318717 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1"} err="failed to get container status \"01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1\": rpc error: code = NotFound desc = could not find container \"01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1\": container with ID starting with 01b696656d3254d3c2f1e7a265506cf5c2df3bbf1f406ebab0fbb20e1babdda1 not found: ID does not exist" Jul 07 20:35:52.318807 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.318735 2553 scope.go:117] "RemoveContainer" containerID="019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc" Jul 07 20:35:52.319039 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:35:52.319014 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc\": container with ID starting with 019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc not found: ID does not exist" containerID="019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc" Jul 07 20:35:52.319124 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.319042 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc"} err="failed to get container status \"019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc\": rpc error: code = NotFound desc = could not find container \"019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc\": container with ID starting with 019b4bc86f463fcc63eb6ed2d0babe9e49a53f49b683d3a8fe541aae92dc7dbc not found: ID does not exist" Jul 07 20:35:52.322550 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.322526 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s"] Jul 07 20:35:52.326145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:52.326123 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c9e405-openshift-default-5db9667f97-pfn6s"] Jul 07 20:35:54.029562 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:54.029528 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83bf22db-e137-4298-a4da-aefd76781fce" path="/var/lib/kubelet/pods/83bf22db-e137-4298-a4da-aefd76781fce/volumes" Jul 07 20:35:55.066330 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.066290 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:55.426261 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.426170 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:55.728358 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728271 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8"] Jul 07 20:35:55.728700 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728677 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" Jul 07 20:35:55.728764 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728738 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" Jul 07 20:35:55.728825 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728767 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" Jul 07 20:35:55.728825 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728777 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" Jul 07 20:35:55.728917 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728892 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" Jul 07 20:35:55.728917 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728902 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b69154-a881-4d41-bde6-0bfa6ae16559" containerName="istio-proxy" Jul 07 20:35:55.728917 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.728910 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="83bf22db-e137-4298-a4da-aefd76781fce" containerName="istio-proxy" Jul 07 20:35:55.732242 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.732223 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.735076 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.735058 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b1597e-openshift-default-dockercfg-nmtdf\"" Jul 07 20:35:55.744620 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.744593 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8"] Jul 07 20:35:55.832372 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832338 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-socket\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832380 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-certs\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832433 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istiod-ca-cert\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832486 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-podinfo\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832522 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bztxf\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-kube-api-access-bztxf\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832564 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-data\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832654 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-envoy\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832682 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-credential-socket\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.832830 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.832767 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-token\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.933880 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.933838 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-token\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934131 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.933906 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-socket\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934131 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.933934 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-certs\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934131 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.933963 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istiod-ca-cert\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934131 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934011 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-podinfo\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934131 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934045 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bztxf\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-kube-api-access-bztxf\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934131 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934081 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-data\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934149 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-envoy\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934177 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-credential-socket\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934449 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934413 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-socket\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934648 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934627 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-credential-socket\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934731 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934679 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-certs\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934731 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934679 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-data\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.934848 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.934824 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istiod-ca-cert\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.936521 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.936496 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-envoy\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.936736 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.936717 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-podinfo\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.942563 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.942536 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-token\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:55.942717 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:55.942700 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bztxf\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-kube-api-access-bztxf\") pod \"e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:56.043611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.043572 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:56.205922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.205880 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8"] Jul 07 20:35:56.206751 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:35:56.206723 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde99491_fa11_4b42_a81e_9a2ff343a3e2.slice/crio-a97418e5c5da707a948a82c9285accd639499599a6c20f97c12f42f304d0d345 WatchSource:0}: Error finding container a97418e5c5da707a948a82c9285accd639499599a6c20f97c12f42f304d0d345: Status 404 returned error can't find the container with id a97418e5c5da707a948a82c9285accd639499599a6c20f97c12f42f304d0d345 Jul 07 20:35:56.209161 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.209130 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:56.209278 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.209208 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:56.209278 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.209253 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:35:56.285138 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.285090 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:56.327125 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.327033 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" event={"ID":"dde99491-fa11-4b42-a81e-9a2ff343a3e2","Type":"ContainerStarted","Data":"927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555"} Jul 07 20:35:56.327125 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.327106 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" event={"ID":"dde99491-fa11-4b42-a81e-9a2ff343a3e2","Type":"ContainerStarted","Data":"a97418e5c5da707a948a82c9285accd639499599a6c20f97c12f42f304d0d345"} Jul 07 20:35:56.349359 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.349310 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:56.349596 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:56.349552 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" podStartSLOduration=1.349535352 podStartE2EDuration="1.349535352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 20:35:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:35:56.347904137 +0000 UTC m=+2108.856576713" watchObservedRunningTime="2026-07-07 20:35:56.349535352 +0000 UTC m=+2108.858207926" Jul 07 20:35:57.043964 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:57.043925 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:57.049106 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:57.049077 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:57.332472 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:57.332389 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:57.333403 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:57.333386 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:35:58.374106 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:58.374070 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:58.476705 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:58.476659 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:58.692922 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:58.692828 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:58.813936 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:58.813894 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:59.057054 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:59.056971 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:35:59.157577 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:35:59.157537 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:36:02.939262 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:02.939216 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:36:08.414603 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:08.414565 2553 generic.go:356] "Generic (PLEG): container finished" podID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerID="927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555" exitCode=0 Jul 07 20:36:08.415060 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:08.414613 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" event={"ID":"dde99491-fa11-4b42-a81e-9a2ff343a3e2","Type":"ContainerDied","Data":"927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555"} Jul 07 20:36:08.415112 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:08.415087 2553 scope.go:117] "RemoveContainer" containerID="927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555" Jul 07 20:36:08.416183 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:08.416152 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:36:08.416258 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:08.416240 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:36:08.416304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:08.416291 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 20:36:09.426312 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:09.426268 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" event={"ID":"dde99491-fa11-4b42-a81e-9a2ff343a3e2","Type":"ContainerStarted","Data":"b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9"} Jul 07 20:36:10.044196 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:10.044161 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:36:10.045152 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:10.045123 2553 patch_prober.go:28] interesting pod/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 07 20:36:10.045281 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:10.045189 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 07 20:36:11.045082 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:11.045039 2553 patch_prober.go:28] interesting pod/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 07 20:36:11.045485 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:11.045134 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 07 20:36:12.044493 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:12.044457 2553 patch_prober.go:28] interesting pod/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 07 20:36:12.044663 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:12.044515 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 07 20:36:12.332747 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:12.332622 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:36:12.816886 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:12.816848 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:36:12.927017 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:12.922840 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:36:13.044930 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:13.044895 2553 patch_prober.go:28] interesting pod/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 07 20:36:13.045120 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:13.044960 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 07 20:36:14.048660 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:14.048632 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:36:14.049709 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:14.049692 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:36:17.011743 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:17.011704 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:36:20.796291 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:20.796250 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:36:20.894832 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:20.894799 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:36:48.245663 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:48.245577 2553 scope.go:117] "RemoveContainer" containerID="dddb1bcac7a1f35cd1b081ba3eaa5d7ac66d5a443c1c213664ffb0b0617f7d51" Jul 07 20:36:57.199938 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:36:57.199901 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:38:57.942643 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:38:57.942604 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:38:58.784543 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:38:58.784503 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:39:14.707318 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:39:14.707279 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:39:30.886525 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:39:30.886471 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:39:50.611702 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:39:50.611627 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:40:05.152437 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:05.152399 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:40:19.681224 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:19.681185 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:40:19.781266 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:19.781232 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:40:47.191893 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:47.191851 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:40:48.090265 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:48.090230 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:40:48.183744 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:48.183717 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:40:48.190890 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:48.190862 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:40:54.855942 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:40:54.855877 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:41:13.566784 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:41:13.566704 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:41:13.666963 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:41:13.666927 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:41:21.563721 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:41:21.563681 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:41:21.666097 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:41:21.666062 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:41:29.563308 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:41:29.563274 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:41:58.814435 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:41:58.814373 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:07.402780 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:07.402744 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:08.179712 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:08.179623 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:09.309216 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:09.309175 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:09.717148 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:09.717056 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:09.814605 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:09.814567 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:09.916584 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:09.916543 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:10.035263 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:10.035228 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:14.794904 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:14.794849 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:20.666505 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:20.666466 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:52.004012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:52.003849 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:52.277575 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:52.277490 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:52.528838 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:52.528752 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:52.819705 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:52.819613 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:53.211608 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:53.211527 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:53.309134 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:53.309101 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:53.410792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:53.410758 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:53.514901 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:53.514864 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:59.790792 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:59.790753 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:42:59.891450 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:42:59.891415 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:43:00.060154 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:00.060061 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8"] Jul 07 20:43:00.060385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:00.060334 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" containerID="cri-o://b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9" gracePeriod=30 Jul 07 20:43:00.070336 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:00.070302 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:43:00.327257 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:00.327153 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:43:00.753611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:00.753571 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bc46d547b-rmn8k_1487b873-3aea-4003-92bc-830301deafaf/authorino/0.log" Jul 07 20:43:05.319728 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.319695 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:43:05.364357 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364268 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-certs\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364357 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364312 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-podinfo\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364357 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364341 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bztxf\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-kube-api-access-bztxf\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364392 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-credential-socket\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364449 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-token\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364474 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-socket\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364498 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istiod-ca-cert\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364554 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-envoy\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364622 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364584 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-data\") pod \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\" (UID: \"dde99491-fa11-4b42-a81e-9a2ff343a3e2\") " Jul 07 20:43:05.364930 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364691 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:43:05.364930 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364746 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:43:05.365064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.364984 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-certs\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.365064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.365025 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-credential-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.365064 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.365041 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 20:43:05.365294 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.365246 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:43:05.366087 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.365970 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:43:05.367212 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.367180 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 20:43:05.367308 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.367211 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 20:43:05.367308 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.367219 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:43:05.367668 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.367646 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-kube-api-access-bztxf" (OuterVolumeSpecName: "kube-api-access-bztxf") pod "dde99491-fa11-4b42-a81e-9a2ff343a3e2" (UID: "dde99491-fa11-4b42-a81e-9a2ff343a3e2"). InnerVolumeSpecName "kube-api-access-bztxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 20:43:05.465824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.465770 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-podinfo\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.465824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.465819 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bztxf\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-kube-api-access-bztxf\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.465824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.465832 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-token\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.465824 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.465844 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-workload-socket\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.466157 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.465851 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istiod-ca-cert\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.466157 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.465860 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-envoy\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.466157 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.465868 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dde99491-fa11-4b42-a81e-9a2ff343a3e2-istio-data\") on node \"ip-10-0-133-86.ec2.internal\" DevicePath \"\"" Jul 07 20:43:05.688594 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.688512 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mn6xh_5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b/manager/0.log" Jul 07 20:43:05.772882 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.772847 2553 generic.go:356] "Generic (PLEG): container finished" podID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerID="b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9" exitCode=0 Jul 07 20:43:05.773101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.772901 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" Jul 07 20:43:05.773101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.772929 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" event={"ID":"dde99491-fa11-4b42-a81e-9a2ff343a3e2","Type":"ContainerDied","Data":"b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9"} Jul 07 20:43:05.773101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.773009 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8" event={"ID":"dde99491-fa11-4b42-a81e-9a2ff343a3e2","Type":"ContainerDied","Data":"a97418e5c5da707a948a82c9285accd639499599a6c20f97c12f42f304d0d345"} Jul 07 20:43:05.773101 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.773041 2553 scope.go:117] "RemoveContainer" containerID="b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9" Jul 07 20:43:05.783521 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.783389 2553 scope.go:117] "RemoveContainer" containerID="927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555" Jul 07 20:43:05.793649 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.793626 2553 scope.go:117] "RemoveContainer" containerID="b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9" Jul 07 20:43:05.793947 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:43:05.793926 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9\": container with ID starting with b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9 not found: ID does not exist" containerID="b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9" Jul 07 20:43:05.794029 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.793959 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9"} err="failed to get container status \"b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9\": rpc error: code = NotFound desc = could not find container \"b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9\": container with ID starting with b341ae4da61b33f5a97c03ab74c82b1bb0062ea64cc66c87841bf52aad40e2b9 not found: ID does not exist" Jul 07 20:43:05.794029 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.793980 2553 scope.go:117] "RemoveContainer" containerID="927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555" Jul 07 20:43:05.794254 ip-10-0-133-86 kubenswrapper[2553]: E0707 20:43:05.794235 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555\": container with ID starting with 927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555 not found: ID does not exist" containerID="927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555" Jul 07 20:43:05.794294 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.794261 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555"} err="failed to get container status \"927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555\": rpc error: code = NotFound desc = could not find container \"927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555\": container with ID starting with 927d9bcf76b3802df825d91970a5a062dbd2ae39f394a53ee7fcd9b9d914a555 not found: ID does not exist" Jul 07 20:43:05.797925 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.797901 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8"] Jul 07 20:43:05.802268 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.802235 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b1597e-openshift-default-5ff9bd9dcf-5hrv8"] Jul 07 20:43:05.811799 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:05.811770 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bqj2k_6db2109b-9c26-410c-ab24-e9c3f4a1c162/manager/0.log" Jul 07 20:43:06.030012 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:06.029967 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" path="/var/lib/kubelet/pods/dde99491-fa11-4b42-a81e-9a2ff343a3e2/volumes" Jul 07 20:43:06.965034 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:06.964979 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-659495cbcb-w4ct6_aa27f1fb-4473-4141-b9ca-5b75edb99ca8/copy-manifests/0.log" Jul 07 20:43:06.972407 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:06.972385 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-659495cbcb-w4ct6_aa27f1fb-4473-4141-b9ca-5b75edb99ca8/manager/0.log" Jul 07 20:43:07.913292 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:07.913258 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc_748bc641-827d-4e44-86a5-0c5685676f1a/pull/0.log" Jul 07 20:43:07.920342 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:07.920304 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc_748bc641-827d-4e44-86a5-0c5685676f1a/extract/0.log" Jul 07 20:43:07.929284 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:07.929256 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc_748bc641-827d-4e44-86a5-0c5685676f1a/util/0.log" Jul 07 20:43:08.054077 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.054046 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9_e10ecf9b-c292-435a-9f4b-e7f61d38b8ce/pull/0.log" Jul 07 20:43:08.062592 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.062572 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9_e10ecf9b-c292-435a-9f4b-e7f61d38b8ce/extract/0.log" Jul 07 20:43:08.070106 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.070085 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9_e10ecf9b-c292-435a-9f4b-e7f61d38b8ce/util/0.log" Jul 07 20:43:08.203945 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.203856 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq_cfc749c5-50dd-4f05-a839-b154db80c33d/extract/0.log" Jul 07 20:43:08.211171 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.211150 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq_cfc749c5-50dd-4f05-a839-b154db80c33d/util/0.log" Jul 07 20:43:08.217921 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.217886 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq_cfc749c5-50dd-4f05-a839-b154db80c33d/pull/0.log" Jul 07 20:43:08.335375 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.335348 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh_018ba219-08fc-46fb-91ec-c0dd64d0dda3/pull/0.log" Jul 07 20:43:08.343756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.343734 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh_018ba219-08fc-46fb-91ec-c0dd64d0dda3/extract/0.log" Jul 07 20:43:08.351510 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.351477 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh_018ba219-08fc-46fb-91ec-c0dd64d0dda3/util/0.log" Jul 07 20:43:08.477040 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.476937 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bc46d547b-rmn8k_1487b873-3aea-4003-92bc-830301deafaf/authorino/0.log" Jul 07 20:43:08.610237 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:08.610201 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qn5fs_00b8a46f-cb14-4201-b20a-0ce726199312/manager/0.log" Jul 07 20:43:09.729734 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:09.729705 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j7j6v_c8753cb8-496d-475b-a348-1099fc543770/istio-proxy/0.log" Jul 07 20:43:10.229593 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:10.229568 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7zsnp_f885ca44-9e68-4b8e-a9a2-e58e58f76b27/istio-proxy/1.log" Jul 07 20:43:10.380157 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:10.380128 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jxth8_6a828525-b750-4c87-a09b-4b8052369d30/payload-pre-processing/0.log" Jul 07 20:43:10.635620 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:10.635538 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-757c6f7d89-5mjdr_a73237eb-e751-46a4-ad48-17242e2dd132/router/0.log" Jul 07 20:43:11.130087 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:11.130056 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-77ljn_9b6b01a9-25ce-4982-ad5d-d005c928122c/main/0.log" Jul 07 20:43:11.253717 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:11.253679 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-65j9v_3606f8af-a164-4a62-815e-9ee64f98eb21/main/0.log" Jul 07 20:43:11.625497 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:11.625463 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-xd75b_6f8be59f-98ca-4ab3-afa8-97448e39fedb/main/0.log" Jul 07 20:43:16.073909 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:16.073866 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:43:16.628437 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:16.628403 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:43:16.689636 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:16.689601 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" limit=1024 Jul 07 20:43:19.828675 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:19.828630 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dsxqc_6e356e3c-7634-4b62-b81d-46cb0a409d97/global-pull-secret-syncer/0.log" Jul 07 20:43:19.965361 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:19.965325 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4d29q_fde5b076-8694-4377-b36d-e5b7656f874f/konnectivity-agent/0.log" Jul 07 20:43:20.085601 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:20.085520 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-86.ec2.internal_e42e328737f94c0a05dc46b508bb8438/haproxy/0.log" Jul 07 20:43:25.096962 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.096927 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc_748bc641-827d-4e44-86a5-0c5685676f1a/extract/0.log" Jul 07 20:43:25.124294 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.124266 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc_748bc641-827d-4e44-86a5-0c5685676f1a/util/0.log" Jul 07 20:43:25.149866 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.149837 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597tbcc_748bc641-827d-4e44-86a5-0c5685676f1a/pull/0.log" Jul 07 20:43:25.178758 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.178729 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9_e10ecf9b-c292-435a-9f4b-e7f61d38b8ce/extract/0.log" Jul 07 20:43:25.203317 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.203281 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9_e10ecf9b-c292-435a-9f4b-e7f61d38b8ce/util/0.log" Jul 07 20:43:25.227899 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.227870 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e082zx9_e10ecf9b-c292-435a-9f4b-e7f61d38b8ce/pull/0.log" Jul 07 20:43:25.256214 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.256186 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq_cfc749c5-50dd-4f05-a839-b154db80c33d/extract/0.log" Jul 07 20:43:25.279927 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.279902 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq_cfc749c5-50dd-4f05-a839-b154db80c33d/util/0.log" Jul 07 20:43:25.304007 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.303969 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736xrxq_cfc749c5-50dd-4f05-a839-b154db80c33d/pull/0.log" Jul 07 20:43:25.336438 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.336409 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh_018ba219-08fc-46fb-91ec-c0dd64d0dda3/extract/0.log" Jul 07 20:43:25.358554 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.358478 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh_018ba219-08fc-46fb-91ec-c0dd64d0dda3/util/0.log" Jul 07 20:43:25.384240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.384213 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18fqzh_018ba219-08fc-46fb-91ec-c0dd64d0dda3/pull/0.log" Jul 07 20:43:25.667651 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.667559 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bc46d547b-rmn8k_1487b873-3aea-4003-92bc-830301deafaf/authorino/0.log" Jul 07 20:43:25.704035 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:25.704003 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qn5fs_00b8a46f-cb14-4201-b20a-0ce726199312/manager/0.log" Jul 07 20:43:27.914749 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:27.914712 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-x9f9x_beb9ec18-4e3a-4655-a6db-6313d35fca26/cluster-monitoring-operator/0.log" Jul 07 20:43:28.170658 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:28.170575 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fttmr_9894d7fb-7f85-4675-97af-639fd731ba0a/node-exporter/0.log" Jul 07 20:43:28.192479 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:28.192454 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fttmr_9894d7fb-7f85-4675-97af-639fd731ba0a/kube-rbac-proxy/0.log" Jul 07 20:43:28.215482 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:28.215453 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fttmr_9894d7fb-7f85-4675-97af-639fd731ba0a/init-textfile/0.log" Jul 07 20:43:28.651431 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:28.651400 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-x98qk_c1e984ed-a4bc-4a76-902d-8e623fb91e42/prometheus-operator-admission-webhook/0.log" Jul 07 20:43:29.987385 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.987342 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn"] Jul 07 20:43:29.987942 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.987920 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" Jul 07 20:43:29.988069 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.987946 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" Jul 07 20:43:29.988069 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.987959 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" Jul 07 20:43:29.988069 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.987966 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" Jul 07 20:43:29.988233 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.988114 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde99491-fa11-4b42-a81e-9a2ff343a3e2" containerName="istio-proxy" Jul 07 20:43:29.991295 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.991272 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:29.994234 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.994210 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww672\"/\"kube-root-ca.crt\"" Jul 07 20:43:29.994349 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.994298 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww672\"/\"openshift-service-ca.crt\"" Jul 07 20:43:29.995437 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:29.995415 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ww672\"/\"default-dockercfg-k44j8\"" Jul 07 20:43:30.002869 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.002844 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn"] Jul 07 20:43:30.091240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.091207 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-sys\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.091240 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.091247 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvz44\" (UniqueName: \"kubernetes.io/projected/f18a3256-2d94-4f90-ab67-df884e6b10d6-kube-api-access-dvz44\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.091480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.091277 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-proc\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.091480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.091378 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-lib-modules\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.091480 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.091440 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-podres\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.159440 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.159407 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-54pwj_a400cf5f-3487-46ce-8351-f46672ac0747/networking-console-plugin/0.log" Jul 07 20:43:30.192145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192103 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-sys\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192145 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192143 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvz44\" (UniqueName: \"kubernetes.io/projected/f18a3256-2d94-4f90-ab67-df884e6b10d6-kube-api-access-dvz44\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192233 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-sys\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192268 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-proc\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192333 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-lib-modules\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192340 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-proc\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192413 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192371 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-podres\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192621 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192479 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-lib-modules\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.192621 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.192492 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f18a3256-2d94-4f90-ab67-df884e6b10d6-podres\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.201774 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.201745 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvz44\" (UniqueName: \"kubernetes.io/projected/f18a3256-2d94-4f90-ab67-df884e6b10d6-kube-api-access-dvz44\") pod \"perf-node-gather-daemonset-4rcmn\" (UID: \"f18a3256-2d94-4f90-ab67-df884e6b10d6\") " pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.301940 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.301832 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.443515 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.443487 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn"] Jul 07 20:43:30.445103 ip-10-0-133-86 kubenswrapper[2553]: W0707 20:43:30.445070 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf18a3256_2d94_4f90_ab67_df884e6b10d6.slice/crio-1d301d9d404dfd2941f1377c01947c76070acd477fb30f9c9f3852ec4f90ebfc WatchSource:0}: Error finding container 1d301d9d404dfd2941f1377c01947c76070acd477fb30f9c9f3852ec4f90ebfc: Status 404 returned error can't find the container with id 1d301d9d404dfd2941f1377c01947c76070acd477fb30f9c9f3852ec4f90ebfc Jul 07 20:43:30.797304 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.797270 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/2.log" Jul 07 20:43:30.801425 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.801396 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-shj6l_b7930caa-d305-4b26-985f-a89a0abb6b08/console-operator/3.log" Jul 07 20:43:30.913441 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.913402 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" event={"ID":"f18a3256-2d94-4f90-ab67-df884e6b10d6","Type":"ContainerStarted","Data":"2f62fc440a3a32022e79d71a36813fbb07c3ab925d30d98a17f63affc621593b"} Jul 07 20:43:30.913589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.913471 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:30.913589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.913493 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" event={"ID":"f18a3256-2d94-4f90-ab67-df884e6b10d6","Type":"ContainerStarted","Data":"1d301d9d404dfd2941f1377c01947c76070acd477fb30f9c9f3852ec4f90ebfc"} Jul 07 20:43:30.932522 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:30.932454 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" podStartSLOduration=1.932437267 podStartE2EDuration="1.932437267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 20:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 20:43:30.930887019 +0000 UTC m=+2563.439559607" watchObservedRunningTime="2026-07-07 20:43:30.932437267 +0000 UTC m=+2563.441109887" Jul 07 20:43:31.391979 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:31.391950 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bbc4f5cf6-wgc57_3c993e07-cbae-4e83-ac06-a6a62f0058b5/console/0.log" Jul 07 20:43:31.426589 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:31.426559 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-pt5cw_6631ff86-3ee8-44d9-8ef5-ba5e2bcd4d41/download-server/0.log" Jul 07 20:43:32.024293 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:32.024262 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-kdtlk_fb2867af-3417-4f9d-9fc6-d4a95a373462/volume-data-source-validator/0.log" Jul 07 20:43:33.174983 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:33.174948 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m5fm_7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b/dns/0.log" Jul 07 20:43:33.197962 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:33.197910 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9m5fm_7b0c8a7b-4eec-40e4-a2bc-563cea6d5a4b/kube-rbac-proxy/0.log" Jul 07 20:43:33.298611 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:33.298574 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t7q6z_2a31e2bb-5d62-4b23-a428-9943115b4cd3/dns-node-resolver/0.log" Jul 07 20:43:33.891352 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:33.891318 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lqwnx_6405eede-1bde-4bf5-b476-ce244e0029eb/node-ca/0.log" Jul 07 20:43:34.894725 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:34.894689 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j7j6v_c8753cb8-496d-475b-a348-1099fc543770/istio-proxy/0.log" Jul 07 20:43:35.087878 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:35.087845 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7zsnp_f885ca44-9e68-4b8e-a9a2-e58e58f76b27/istio-proxy/0.log" Jul 07 20:43:35.115411 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:35.115378 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7zsnp_f885ca44-9e68-4b8e-a9a2-e58e58f76b27/istio-proxy/1.log" Jul 07 20:43:35.147213 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:35.147121 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jxth8_6a828525-b750-4c87-a09b-4b8052369d30/payload-pre-processing/0.log" Jul 07 20:43:35.218316 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:35.218288 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-757c6f7d89-5mjdr_a73237eb-e751-46a4-ad48-17242e2dd132/router/0.log" Jul 07 20:43:35.939909 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:35.939880 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4plnv_b2ca5bd3-de4d-43a0-b8f9-49d0e24b1b03/serve-healthcheck-canary/0.log" Jul 07 20:43:36.488889 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:36.488854 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-cqz4g_d2bcb0b9-8e9b-4cd1-9633-853e6668363e/insights-operator/0.log" Jul 07 20:43:36.490621 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:36.490599 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-cqz4g_d2bcb0b9-8e9b-4cd1-9633-853e6668363e/insights-operator/1.log" Jul 07 20:43:36.583220 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:36.583191 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s6npk_08eeb706-817f-4562-859c-f42a542511a3/kube-rbac-proxy/0.log" Jul 07 20:43:36.607532 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:36.607506 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s6npk_08eeb706-817f-4562-859c-f42a542511a3/exporter/0.log" Jul 07 20:43:36.631709 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:36.631677 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s6npk_08eeb706-817f-4562-859c-f42a542511a3/extractor/0.log" Jul 07 20:43:36.929583 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:36.929515 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ww672/perf-node-gather-daemonset-4rcmn" Jul 07 20:43:39.005691 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:39.005658 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mn6xh_5bbd2b4a-8d41-4849-bfe1-bbe11dd6616b/manager/0.log" Jul 07 20:43:39.140929 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:39.140900 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bqj2k_6db2109b-9c26-410c-ab24-e9c3f4a1c162/manager/0.log" Jul 07 20:43:39.626640 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:39.626606 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-659495cbcb-w4ct6_aa27f1fb-4473-4141-b9ca-5b75edb99ca8/manager/0.log" Jul 07 20:43:39.652504 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:39.652431 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-659495cbcb-w4ct6_aa27f1fb-4473-4141-b9ca-5b75edb99ca8/copy-manifests/0.log" Jul 07 20:43:40.355913 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:40.355879 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j7j6v_c8753cb8-496d-475b-a348-1099fc543770/istio-proxy/0.log" Jul 07 20:43:40.510717 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:40.510686 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7zsnp_f885ca44-9e68-4b8e-a9a2-e58e58f76b27/istio-proxy/0.log" Jul 07 20:43:40.537044 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:40.537009 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7zsnp_f885ca44-9e68-4b8e-a9a2-e58e58f76b27/istio-proxy/1.log" Jul 07 20:43:40.569408 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:40.569378 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jxth8_6a828525-b750-4c87-a09b-4b8052369d30/payload-pre-processing/0.log" Jul 07 20:43:40.619276 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:40.619245 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-757c6f7d89-5mjdr_a73237eb-e751-46a4-ad48-17242e2dd132/router/0.log" Jul 07 20:43:41.082108 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:41.082080 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-699bddd6bc-hx5gd_f815a95f-1826-409e-9b70-ee21022297c7/manager/0.log" Jul 07 20:43:41.849400 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:41.849310 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4j7j6v_c8753cb8-496d-475b-a348-1099fc543770/istio-proxy/0.log" Jul 07 20:43:41.998756 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:41.998722 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7zsnp_f885ca44-9e68-4b8e-a9a2-e58e58f76b27/istio-proxy/0.log" Jul 07 20:43:42.032862 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:42.032822 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7zsnp_f885ca44-9e68-4b8e-a9a2-e58e58f76b27/istio-proxy/1.log" Jul 07 20:43:42.054975 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:42.054939 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jxth8_6a828525-b750-4c87-a09b-4b8052369d30/payload-pre-processing/0.log" Jul 07 20:43:42.119849 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:42.119778 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-757c6f7d89-5mjdr_a73237eb-e751-46a4-ad48-17242e2dd132/router/0.log" Jul 07 20:43:46.533453 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:46.533410 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-8795t_d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a/kube-storage-version-migrator-operator/1.log" Jul 07 20:43:46.534273 ip-10-0-133-86 kubenswrapper[2553]: I0707 20:43:46.534257 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-8795t_d3fcf2c5-7c84-4931-b2f7-b64c956b4a0a/kube-storage-version-migrator-operator/0.log"