Jul 12 14:39:10.771748 ip-10-0-142-78 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 12 14:39:10.771758 ip-10-0-142-78 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 12 14:39:10.771765 ip-10-0-142-78 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 12 14:39:10.771995 ip-10-0-142-78 systemd[1]: Failed to start Kubernetes Kubelet. Jul 12 14:39:20.943841 ip-10-0-142-78 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 12 14:39:20.943862 ip-10-0-142-78 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ffe38bed7f80478ebb1294d8f92c97bd -- Jul 12 14:40:50.487736 ip-10-0-142-78 systemd[1]: Starting Kubernetes Kubelet... Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.940952 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945023 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945034 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945038 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945041 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945044 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945047 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945052 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 14:40:50.949229 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945055 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945058 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945061 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945063 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945066 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945069 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945071 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945074 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945077 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945079 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945082 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945084 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945087 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945090 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945093 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945096 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945098 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945103 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945107 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 14:40:50.956800 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945110 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945113 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945115 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945118 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945121 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945123 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945126 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945129 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945131 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945134 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945136 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945139 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945141 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945144 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945146 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945148 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945153 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945157 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945160 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945163 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 14:40:50.961506 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945165 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945168 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945170 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945172 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945175 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945178 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945182 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945186 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945189 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945192 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945195 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945198 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945201 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945203 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945206 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945208 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945211 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945213 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945216 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 14:40:50.979744 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945219 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945228 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945231 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945234 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945236 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945239 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945241 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945243 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945246 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945250 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945252 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945254 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945257 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945259 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945262 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945264 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945267 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945269 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945272 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945274 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 14:40:50.981779 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945277 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945280 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945283 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945286 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945289 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945291 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945294 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945296 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945298 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945301 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945304 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945306 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945308 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.945311 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946059 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946066 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946069 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946072 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946075 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946077 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 14:40:50.983866 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946080 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946083 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946085 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946087 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946090 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946092 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946095 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946097 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946100 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946102 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946106 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946109 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946113 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946115 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946118 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946120 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946123 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946126 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946128 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946131 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 14:40:50.999588 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946133 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946136 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946139 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946141 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946144 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946146 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946149 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946152 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946155 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946157 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946160 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946162 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946165 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946167 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946170 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946172 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946175 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946177 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946179 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 14:40:51.016788 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946182 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946185 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946189 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946192 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946194 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946197 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946200 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946202 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946205 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946207 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946210 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946212 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946214 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946219 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946223 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946226 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946229 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946232 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946235 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 14:40:51.031656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946238 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 14:40:51.020322 ip-10-0-142-78 systemd[1]: Started Kubernetes Kubelet. Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946240 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946244 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946247 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946249 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946252 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946256 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946258 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946260 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946263 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946265 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946268 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946271 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946273 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946275 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946278 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946280 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946284 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946287 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 14:40:51.048843 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946290 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946293 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946296 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946298 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946300 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946303 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946306 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946308 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946311 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946314 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946316 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946319 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946321 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946324 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946327 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946329 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946721 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946727 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946730 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946733 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 14:40:51.064571 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946736 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946738 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946741 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946743 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946746 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946748 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946751 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946754 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946756 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946758 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946760 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946763 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946765 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946768 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946770 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946773 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946775 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946778 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946780 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946783 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 14:40:51.097728 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946786 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946788 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946791 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946794 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946796 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946799 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946804 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946806 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946810 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946812 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946815 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946817 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946820 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946822 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946825 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946827 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946830 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946833 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946835 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946838 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 14:40:51.249507 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946840 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946842 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946846 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946849 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946852 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946855 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946858 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946861 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946863 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946866 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946868 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946871 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946875 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946880 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946884 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946887 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946889 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946892 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946896 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 14:40:51.250930 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946899 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946901 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946904 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946907 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946909 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946912 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946914 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946917 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946919 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946921 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946924 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946926 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946929 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946932 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946934 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946936 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946939 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946941 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946943 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946946 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 14:40:51.251743 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946948 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946951 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946953 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946956 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946959 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946963 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946966 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946968 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946970 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946973 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946975 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946979 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946982 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946984 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946987 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.946990 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947067 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947077 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947087 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947094 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947103 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 14:40:51.252998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947107 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947111 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947115 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947118 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947121 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947124 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947127 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947130 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947133 2573 flags.go:64] FLAG: --cgroup-root="" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947136 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947139 2573 flags.go:64] FLAG: --client-ca-file="" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947142 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947145 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947148 2573 flags.go:64] FLAG: --cluster-domain="" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947151 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947154 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947159 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947163 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947166 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947169 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947172 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947175 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947180 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947183 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 14:40:51.253792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947186 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947189 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947192 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947197 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947200 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947203 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947206 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947208 2573 flags.go:64] FLAG: --enable-server="true" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947211 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947220 2573 flags.go:64] FLAG: --event-burst="100" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947223 2573 flags.go:64] FLAG: --event-qps="50" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947226 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947229 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947232 2573 flags.go:64] FLAG: --eviction-hard="" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947237 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947240 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947242 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947245 2573 flags.go:64] FLAG: --eviction-soft="" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947248 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947252 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947255 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947258 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947260 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947263 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947267 2573 flags.go:64] FLAG: --feature-gates="" Jul 12 14:40:51.254561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947271 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947273 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947276 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947279 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947282 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947287 2573 flags.go:64] FLAG: --help="false" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947290 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947293 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947295 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947298 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947302 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947306 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947308 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947311 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947314 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947317 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947320 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947323 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947325 2573 flags.go:64] FLAG: --kube-reserved="" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947328 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947331 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947334 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947337 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947339 2573 flags.go:64] FLAG: --lock-file="" Jul 12 14:40:51.255403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947342 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947345 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947348 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947353 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947356 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947359 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947362 2573 flags.go:64] FLAG: --logging-format="text" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947366 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947370 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947372 2573 flags.go:64] FLAG: --manifest-url="" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947375 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947379 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947382 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947388 2573 flags.go:64] FLAG: --max-pods="110" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947391 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947394 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947397 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947400 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947403 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947405 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947422 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947430 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947433 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947436 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 14:40:51.256227 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947439 2573 flags.go:64] FLAG: --pod-cidr="" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947442 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947448 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947450 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947453 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947456 2573 flags.go:64] FLAG: --port="10250" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947459 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947463 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f229ca9c39144bc3" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947466 2573 flags.go:64] FLAG: --qos-reserved="" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947469 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947472 2573 flags.go:64] FLAG: --register-node="true" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947474 2573 flags.go:64] FLAG: --register-with-taints="" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947478 2573 flags.go:64] FLAG: --registry-burst="10" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947481 2573 flags.go:64] FLAG: --registry-qps="5" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947483 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947486 2573 flags.go:64] FLAG: --reserved-memory="" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947493 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947496 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947499 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947502 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947505 2573 flags.go:64] FLAG: --runonce="false" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947509 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947512 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947515 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947518 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 14:40:51.257336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947520 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947523 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947526 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947529 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947533 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947536 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947539 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947541 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947544 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947547 2573 flags.go:64] FLAG: --system-cgroups="" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947549 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947555 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947557 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947560 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947564 2573 flags.go:64] FLAG: --tls-min-version="" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947566 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947569 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947572 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947574 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947577 2573 flags.go:64] FLAG: --v="2" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947581 2573 flags.go:64] FLAG: --version="false" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947584 2573 flags.go:64] FLAG: --vmodule="" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947588 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.947592 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947724 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 14:40:51.258359 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947728 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947731 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947734 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947738 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947741 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947744 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947747 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947749 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947751 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947754 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947757 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947760 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947762 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947765 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947768 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947770 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947773 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947775 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947777 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947780 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 14:40:51.259150 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947782 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947785 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947787 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947789 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947792 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947794 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947797 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947799 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947802 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947804 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947808 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947811 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947813 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947816 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947819 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947825 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947828 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947830 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947833 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947835 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 14:40:51.259760 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947838 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947840 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947843 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947845 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947848 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947851 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947853 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947855 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947858 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947860 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947863 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947865 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947868 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947870 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947872 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947875 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947877 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947880 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947882 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947885 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 14:40:51.260326 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947887 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947890 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947893 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947896 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947899 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947901 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947903 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947911 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947915 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947918 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947921 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947923 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947926 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947928 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947931 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947933 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947936 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947938 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947941 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947944 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 14:40:51.261118 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947946 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947949 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947953 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947956 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947959 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947961 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947964 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947967 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947969 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947972 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947974 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947977 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947979 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947981 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947985 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947988 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947990 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 14:40:51.261911 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.947992 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.948779 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.958442 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.958463 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.959343 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.959354 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959433 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959440 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959443 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959446 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959449 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959452 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959454 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959457 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 14:40:51.262738 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959460 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959463 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959465 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959468 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959470 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959474 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959476 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959479 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959481 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959483 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959486 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959489 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959491 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959493 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959496 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959499 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959502 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959504 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959507 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959511 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 14:40:51.263437 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959514 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959516 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959519 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959521 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959523 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959526 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959529 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959531 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959533 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959536 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959538 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959541 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959543 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959546 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959548 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959550 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959553 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959555 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959558 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959560 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 14:40:51.265980 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959563 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959566 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959568 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959571 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959574 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959576 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959579 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959582 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959584 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959586 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959589 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959596 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959600 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959603 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959605 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959608 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959610 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959614 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959617 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 14:40:51.267751 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959621 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959626 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959630 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959633 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959636 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959639 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959642 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959644 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959647 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959650 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959652 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959655 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959658 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959661 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959664 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959667 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959669 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959672 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959674 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 14:40:51.270387 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959676 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959679 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959681 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959684 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959686 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959691 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959694 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959697 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959699 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959702 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959705 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959708 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959710 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.959716 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959836 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 14:40:51.272340 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959842 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959845 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959847 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959850 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959853 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959856 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959859 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959862 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959864 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959867 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959870 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959873 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959876 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959878 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959881 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959883 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959886 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959888 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959891 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 14:40:51.274925 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959893 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959896 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959898 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959901 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959905 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959907 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959910 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959913 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959915 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959918 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959920 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959923 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959928 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959932 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959935 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959938 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959940 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959943 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959946 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959949 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 14:40:51.277331 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959952 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959954 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959956 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959959 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959961 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959965 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959967 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959970 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959972 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959974 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959977 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959980 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959982 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959986 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959989 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959992 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959995 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.959998 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960000 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 14:40:51.279088 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960003 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960005 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960008 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960010 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960013 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960016 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960019 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960023 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960026 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960029 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960032 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960034 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960036 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960039 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960041 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960044 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960046 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960049 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960051 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 14:40:51.279922 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960054 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960057 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960059 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960062 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960064 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960066 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960069 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960071 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960074 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960076 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960080 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960082 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960085 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960087 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960090 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960092 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960094 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960097 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960101 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960104 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 14:40:51.281079 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:50.960106 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.960111 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.960239 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.962599 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.964192 2573 server.go:1019] "Starting client certificate rotation" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.964294 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.964331 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.967211 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.968346 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.978539 2573 log.go:25] "Validated CRI v1 runtime API" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.980849 2573 log.go:25] "Validated CRI v1 image API" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.981775 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 14:40:51.281679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.986348 2573 fs.go:135] Filesystem UUIDs: map[0b36bb35-cab9-4401-ba8a-94fe18568000:/dev/nvme0n1p3 557f7e58-ba5d-4387-9569-e3d6bbc2bd05:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 12 14:40:51.282272 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.986363 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.993945 2573 manager.go:216] Machine: {Timestamp:2026-07-12 14:40:50.992198931 +0000 UTC m=+0.377683073 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200109 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26cc2e6248e7bd073aed1ffb66bf30 SystemUUID:ec26cc2e-6248-e7bd-073a-ed1ffb66bf30 BootID:ffe38bed-7f80-478e-bb12-94d8f92c97bd Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fa:70:9c:8e:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fa:70:9c:8e:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:75:c1:ba:66:04 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.994062 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.994146 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.997268 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.997294 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-78.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.998044 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.998057 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 14:40:51.282676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.998065 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.998835 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:50.999721 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.000503 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.004841 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.004958 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.004986 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.004997 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.005011 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.005020 2573 kubelet.go:398] "Adding apiserver pod source" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.005028 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.005391 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.005407 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.008663 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011074 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011102 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011246 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011257 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011263 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011268 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011276 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011281 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011914 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011921 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011929 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011935 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011943 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.011951 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.012849 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.012856 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.017467 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-78.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.017520 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.017523 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.019318 2573 server.go:1268] "Started kubelet" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.019387 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.019401 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.019488 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.020904 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 14:40:51.283105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.022471 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.024135 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-78.ec2.internal.18c191d03f751643 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-78.ec2.internal,UID:ip-10-0-142-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-78.ec2.internal,},FirstTimestamp:2026-07-12 14:40:51.019290179 +0000 UTC m=+0.404774319,LastTimestamp:2026-07-12 14:40:51.019290179 +0000 UTC m=+0.404774319,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-78.ec2.internal,}" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.025565 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.031587 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.031600 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.032537 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033247 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033248 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033269 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033388 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033399 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033429 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033501 2573 factory.go:55] Registering systemd factory Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.033511 2573 factory.go:223] Registration of the systemd container factory successfully Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.034111 2573 factory.go:153] Registering CRI-O factory Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.034135 2573 factory.go:223] Registration of the crio container factory successfully Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.034153 2573 factory.go:103] Registering Raw factory Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.034163 2573 manager.go:1196] Started watching for new ooms in manager Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.035477 2573 manager.go:349] Starting recovery of all containers Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.043970 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jkpmw" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.044148 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.044168 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.045810 2573 manager.go:354] Recovery completed Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.050738 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.051567 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jkpmw" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.052488 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientMemory" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.052513 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.052523 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientPID" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.053004 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.053014 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.053028 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.053035 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 14:40:51.284123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.053673 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientMemory" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.053695 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.053704 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientPID" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.055019 2573 policy_none.go:49] "None policy: Start" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.055033 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.055042 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.056862 2573 policy_none.go:47] "Start" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.119938 2573 manager.go:337] "Starting Device Plugin manager" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.120117 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.120134 2573 server.go:87] "Starting device plugin registration server" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.120459 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.120473 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.120619 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.120703 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.120711 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.121200 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.121257 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.150744 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.152048 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.152080 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.152102 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.152138 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.154505 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.221452 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.222478 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientMemory" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.222508 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.222521 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientPID" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.222548 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.234578 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.234605 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-78.ec2.internal\": node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.252368 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal"] Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.252461 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.253572 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientMemory" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.253602 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.253614 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientPID" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.255782 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.255881 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.255912 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 14:40:51.285017 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.256504 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientMemory" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.256543 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.256559 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientPID" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.256607 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientMemory" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.256626 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.256640 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientPID" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.258766 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.258788 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.259407 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientMemory" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.259431 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.259450 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.259461 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeHasSufficientPID" Jul 12 14:40:51.286053 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:51.266761 2573 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d2ac267c25452ab1b32df9632db77.slice/cpu.weight": open /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d2ac267c25452ab1b32df9632db77.slice/cpu.weight: no such device Jul 12 14:40:51.417439 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.313308 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-78.ec2.internal\" not found" node="ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.417439 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.317596 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-78.ec2.internal\" not found" node="ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.417439 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.335381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a94ea4805b79c488e4179fc1621147fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal\" (UID: \"a94ea4805b79c488e4179fc1621147fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.417439 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.335426 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a94ea4805b79c488e4179fc1621147fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal\" (UID: \"a94ea4805b79c488e4179fc1621147fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.417439 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.335493 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42d2ac267c25452ab1b32df9632db77-config\") pod \"kube-apiserver-proxy-ip-10-0-142-78.ec2.internal\" (UID: \"e42d2ac267c25452ab1b32df9632db77\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.417439 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.360343 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.437175 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.435759 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a94ea4805b79c488e4179fc1621147fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal\" (UID: \"a94ea4805b79c488e4179fc1621147fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.437175 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.435725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a94ea4805b79c488e4179fc1621147fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal\" (UID: \"a94ea4805b79c488e4179fc1621147fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.437175 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.435852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a94ea4805b79c488e4179fc1621147fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal\" (UID: \"a94ea4805b79c488e4179fc1621147fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.437175 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.435871 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42d2ac267c25452ab1b32df9632db77-config\") pod \"kube-apiserver-proxy-ip-10-0-142-78.ec2.internal\" (UID: \"e42d2ac267c25452ab1b32df9632db77\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.437175 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.435897 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42d2ac267c25452ab1b32df9632db77-config\") pod \"kube-apiserver-proxy-ip-10-0-142-78.ec2.internal\" (UID: \"e42d2ac267c25452ab1b32df9632db77\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.437175 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.435926 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a94ea4805b79c488e4179fc1621147fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal\" (UID: \"a94ea4805b79c488e4179fc1621147fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.540097 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.460852 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.664609 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.561676 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.664609 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.615829 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.664609 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.620253 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:51.664609 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.662030 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.792920 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.762518 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.907357 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.862930 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.963531 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:51.963442 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:51.963531 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.963476 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 14:40:51.964052 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:51.963636 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 14:40:53.441774 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.441754 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 14:40:53.474181 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.474134 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 14:35:52 +0000 UTC" deadline="2028-04-23 10:39:32.733222162 +0000 UTC" Jul 12 14:40:53.474181 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.474170 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15619h58m39.259055077s" Jul 12 14:40:53.481314 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:53.481286 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:53.582234 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:53.582205 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-78.ec2.internal\" not found" Jul 12 14:40:53.621647 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.621613 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 14:40:53.626151 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.626130 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 14:40:53.650742 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.650713 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" Jul 12 14:40:53.662259 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.662236 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 14:40:53.663413 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.663400 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" Jul 12 14:40:53.684275 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.684249 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 14:40:53.769199 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:53.769167 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42d2ac267c25452ab1b32df9632db77.slice/crio-92ea365a5c858c5a3a58e09b38740db4b54b1d7807619041c02f2f8edd1c327a WatchSource:0}: Error finding container 92ea365a5c858c5a3a58e09b38740db4b54b1d7807619041c02f2f8edd1c327a: Status 404 returned error can't find the container with id 92ea365a5c858c5a3a58e09b38740db4b54b1d7807619041c02f2f8edd1c327a Jul 12 14:40:53.769805 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:53.769783 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94ea4805b79c488e4179fc1621147fc.slice/crio-48c6cdda36d1feff85d819a76608b173b48187c08663ea0a2f27ead0245a5b17 WatchSource:0}: Error finding container 48c6cdda36d1feff85d819a76608b173b48187c08663ea0a2f27ead0245a5b17: Status 404 returned error can't find the container with id 48c6cdda36d1feff85d819a76608b173b48187c08663ea0a2f27ead0245a5b17 Jul 12 14:40:53.773882 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:53.773867 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 14:40:54.424106 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.424075 2573 apiserver.go:52] "Watching apiserver" Jul 12 14:40:54.429983 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.429804 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 14:40:54.432260 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.431851 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-567hq","openshift-multus/multus-rkmbl","openshift-network-operator/iptables-alerter-r86t9","openshift-ovn-kubernetes/ovnkube-node-hrksg","kube-system/konnectivity-agent-2pmbg","kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x","openshift-cluster-node-tuning-operator/tuned-9n9h2","openshift-image-registry/node-ca-6mbxr","openshift-multus/network-metrics-daemon-fjw4t","openshift-network-diagnostics/network-check-target-2pxd8","kube-system/global-pull-secret-syncer-jbkhl","openshift-dns/node-resolver-ppcrh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal"] Jul 12 14:40:54.434377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.434353 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.436749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.436671 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 14:40:54.436998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.436975 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-twmxl\"" Jul 12 14:40:54.437583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.437180 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 14:40:54.437583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.437266 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.437583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.437362 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 14:40:54.438784 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.438765 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.440426 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.440394 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cl7qk\"" Jul 12 14:40:54.440565 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.440544 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 14:40:54.440723 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.440707 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 14:40:54.441037 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.441004 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 14:40:54.441119 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.441074 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 14:40:54.441221 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.441192 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 14:40:54.441374 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.441359 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gkq5t\"" Jul 12 14:40:54.441751 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.441733 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.443868 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.443847 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 14:40:54.444030 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.443887 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 14:40:54.444030 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.443945 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tzb47\"" Jul 12 14:40:54.444030 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.443965 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.444030 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.443891 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 14:40:54.445202 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.445139 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 14:40:54.445370 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.445295 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 14:40:54.445370 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.445298 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 14:40:54.445836 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.445818 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 14:40:54.446212 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.446195 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 14:40:54.446506 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.446489 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.449712 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.449539 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 14:40:54.452497 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.450197 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d2bmg\"" Jul 12 14:40:54.452497 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.451911 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f9fwp\"" Jul 12 14:40:54.452497 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.452150 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 14:40:54.452497 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.452196 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 14:40:54.454794 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.454777 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.456745 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.456670 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 14:40:54.456948 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.456932 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 14:40:54.457260 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.457158 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 14:40:54.457663 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.457648 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.458603 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.458558 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k6rgx\"" Jul 12 14:40:54.460329 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.460304 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 14:40:54.460419 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.460386 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.460477 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.460311 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 14:40:54.460527 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.460496 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 14:40:54.460806 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.460789 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 14:40:54.460996 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.460979 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 14:40:54.461248 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.461232 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c7k6p\"" Jul 12 14:40:54.461722 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.461701 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 14:40:54.462129 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462101 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-env-overrides\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462211 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462137 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovn-node-metrics-cert\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462211 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a63be01d-18d1-40e1-8864-fb315b971072-konnectivity-ca\") pod \"konnectivity-agent-2pmbg\" (UID: \"a63be01d-18d1-40e1-8864-fb315b971072\") " pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.462211 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462104 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 14:40:54.462356 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462203 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-modprobe-d\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462356 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462240 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-tmp\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462356 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462267 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-registration-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.462356 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-iptables-alerter-script\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.462550 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462366 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-node-log\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462550 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462400 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-cni-bin\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462550 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462428 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovnkube-config\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462550 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462464 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-socket-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.462550 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdqlz\" (UniqueName: \"kubernetes.io/projected/c1140271-ad4b-4cd7-96e8-64318c687921-kube-api-access-zdqlz\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.462550 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462512 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crf7v\" (UniqueName: \"kubernetes.io/projected/7aa6132f-7dbf-439b-8683-2d1aca184a98-kube-api-access-crf7v\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-kubelet\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462575 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovnkube-script-lib\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462597 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t2vd\" (UniqueName: \"kubernetes.io/projected/2b612495-6f09-40f5-9eb9-b8e603d61d37-kube-api-access-7t2vd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-device-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462641 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-sys\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462667 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysconfig\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462689 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-systemd\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-run\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462730 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-systemd-units\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462751 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysctl-conf\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462785 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-tuned\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8czt6\" (UniqueName: \"kubernetes.io/projected/3043ff23-e0c3-4c6a-80a0-80566bc79d76-kube-api-access-8czt6\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462839 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.462860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-systemd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-ovn\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462906 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-log-socket\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462929 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-run-ovn-kubernetes\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462951 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a63be01d-18d1-40e1-8864-fb315b971072-agent-certs\") pod \"konnectivity-agent-2pmbg\" (UID: \"a63be01d-18d1-40e1-8864-fb315b971072\") " pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.462974 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7frr\" (UniqueName: \"kubernetes.io/projected/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-kube-api-access-x7frr\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463004 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7aa6132f-7dbf-439b-8683-2d1aca184a98-hosts-file\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aa6132f-7dbf-439b-8683-2d1aca184a98-tmp-dir\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-host-slash\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-slash\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-run-netns\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-etc-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysctl-d\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-lib-modules\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.463549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463206 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-var-lib-kubelet\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463228 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3043ff23-e0c3-4c6a-80a0-80566bc79d76-host\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463247 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3043ff23-e0c3-4c6a-80a0-80566bc79d76-serviceca\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463269 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cskmg\" (UniqueName: \"kubernetes.io/projected/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-kube-api-access-cskmg\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463322 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-cni-netd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463342 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-etc-selinux\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463362 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-kubernetes\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463383 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-host\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463407 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-sys-fs\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463431 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-var-lib-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463457 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.464319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.463840 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ckrwc\"" Jul 12 14:40:54.465004 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.464984 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:54.465199 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.465076 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:40:54.467403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.467033 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:54.467403 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.467109 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:40:54.469556 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.469464 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.469556 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.469522 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:40:54.495140 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.494925 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sjknm" Jul 12 14:40:54.503794 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.503766 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sjknm" Jul 12 14:40:54.551979 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.551952 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 14:40:54.564208 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.564175 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t2vd\" (UniqueName: \"kubernetes.io/projected/2b612495-6f09-40f5-9eb9-b8e603d61d37-kube-api-access-7t2vd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.564382 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.564235 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-system-cni-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.564382 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.564278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovnkube-config\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.564520 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.564505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:54.564578 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.564545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysconfig\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.564578 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.564570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-systemd\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.564670 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.564598 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-systemd-units\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571130 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571100 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-daemon-config\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.571288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571156 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysctl-conf\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.571288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571221 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-tuned\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.571288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8czt6\" (UniqueName: \"kubernetes.io/projected/3043ff23-e0c3-4c6a-80a0-80566bc79d76-kube-api-access-8czt6\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.571288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571275 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-systemd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.571509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571336 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-kubelet\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.571509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571364 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-etc-kubernetes\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.571509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571394 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:54.571509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571419 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571448 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a63be01d-18d1-40e1-8864-fb315b971072-agent-certs\") pod \"konnectivity-agent-2pmbg\" (UID: \"a63be01d-18d1-40e1-8864-fb315b971072\") " pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.571509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7frr\" (UniqueName: \"kubernetes.io/projected/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-kube-api-access-x7frr\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-host-slash\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571539 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-slash\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-etc-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysctl-d\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571622 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3043ff23-e0c3-4c6a-80a0-80566bc79d76-host\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571649 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cskmg\" (UniqueName: \"kubernetes.io/projected/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-kube-api-access-cskmg\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571689 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571719 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-env-overrides\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571747 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovn-node-metrics-cert\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.571822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-netns\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571833 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-multus-certs\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571863 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-etc-selinux\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571893 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-kubernetes\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571922 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-host\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571960 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-sys-fs\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.571994 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-var-lib-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572044 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-tuning-conf-dir\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572073 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-socket-dir-parent\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572104 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-cni-bin\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572137 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a63be01d-18d1-40e1-8864-fb315b971072-konnectivity-ca\") pod \"konnectivity-agent-2pmbg\" (UID: \"a63be01d-18d1-40e1-8864-fb315b971072\") " pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572171 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-modprobe-d\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572202 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-registration-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572228 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-cnibin\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.572288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-cni-binary-copy\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572289 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-cni-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572341 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-socket-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572371 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovnkube-script-lib\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572402 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-os-release\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572434 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-conf-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrx7\" (UniqueName: \"kubernetes.io/projected/903d8cfd-d90e-42ca-8a0e-54b2763363ac-kube-api-access-qnrx7\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-device-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-sys\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572558 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-run\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572588 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-ovn\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572621 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-run-ovn-kubernetes\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572653 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx7qw\" (UniqueName: \"kubernetes.io/projected/9ab38523-f4fb-4fb3-9111-4dd8070e71de-kube-api-access-rx7qw\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ee45648f-77ef-4944-aa18-e1c690675ef4-kubelet-config\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.572941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572713 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-log-socket\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-cnibin\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572833 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/903d8cfd-d90e-42ca-8a0e-54b2763363ac-cni-binary-copy\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-hostroot\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7aa6132f-7dbf-439b-8683-2d1aca184a98-hosts-file\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aa6132f-7dbf-439b-8683-2d1aca184a98-tmp-dir\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572967 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-run-netns\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.572998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-os-release\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573057 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-lib-modules\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-var-lib-kubelet\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3043ff23-e0c3-4c6a-80a0-80566bc79d76-serviceca\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573164 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-cni-netd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.573589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573196 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-k8s-cni-cncf-io\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573229 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdc8f\" (UniqueName: \"kubernetes.io/projected/0e85603c-a47b-4425-b415-f66384553038-kube-api-access-fdc8f\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-cni-multus\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573289 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-tmp\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573324 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-iptables-alerter-script\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573359 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-system-cni-dir\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-node-log\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-cni-bin\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ee45648f-77ef-4944-aa18-e1c690675ef4-dbus\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdqlz\" (UniqueName: \"kubernetes.io/projected/c1140271-ad4b-4cd7-96e8-64318c687921-kube-api-access-zdqlz\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crf7v\" (UniqueName: \"kubernetes.io/projected/7aa6132f-7dbf-439b-8683-2d1aca184a98-kube-api-access-crf7v\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573588 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-kubelet\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573705 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-kubelet\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.574467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.573869 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysctl-conf\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.575146 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.574226 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 14:40:54.575146 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.574742 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.575283 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575260 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovnkube-script-lib\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.575388 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575370 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7aa6132f-7dbf-439b-8683-2d1aca184a98-tmp-dir\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.575736 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575716 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-etc-selinux\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.575818 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575783 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-kubernetes\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.576387 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575875 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-ovn\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.576387 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575924 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-run-ovn-kubernetes\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.576387 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575935 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-run\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.576387 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.575979 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-host\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.576387 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576049 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-run-systemd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.576387 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576063 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-sys-fs\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.576387 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576172 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-socket-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.576774 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576571 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a63be01d-18d1-40e1-8864-fb315b971072-konnectivity-ca\") pod \"konnectivity-agent-2pmbg\" (UID: \"a63be01d-18d1-40e1-8864-fb315b971072\") " pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.576774 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576679 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-modprobe-d\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.576774 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576721 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-registration-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.576774 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576762 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-var-lib-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.576967 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576802 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-device-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.576967 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.576838 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-sys\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.578411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.577196 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-slash\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.578411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.577240 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-etc-openvswitch\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.578411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.577272 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-host-slash\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.578411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.577352 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysctl-d\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.578411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.577384 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3043ff23-e0c3-4c6a-80a0-80566bc79d76-host\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.578411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.577620 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.578411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.578003 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-env-overrides\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.579318 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.579294 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovnkube-config\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.579870 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.579852 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1140271-ad4b-4cd7-96e8-64318c687921-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.579937 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.579911 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-log-socket\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.579989 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.579971 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-run-netns\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.580134 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580118 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-lib-modules\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.580215 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580198 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-var-lib-kubelet\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.580606 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580583 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3043ff23-e0c3-4c6a-80a0-80566bc79d76-serviceca\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.580689 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580664 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-cni-netd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.580689 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580664 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a63be01d-18d1-40e1-8864-fb315b971072-agent-certs\") pod \"konnectivity-agent-2pmbg\" (UID: \"a63be01d-18d1-40e1-8864-fb315b971072\") " pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.580838 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580821 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-sysconfig\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.580899 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580882 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-host-cni-bin\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.581667 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.580957 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-systemd\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.581667 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.581098 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-node-log\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.581667 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.581158 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7aa6132f-7dbf-439b-8683-2d1aca184a98-hosts-file\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.581875 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.581660 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b612495-6f09-40f5-9eb9-b8e603d61d37-systemd-units\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.584981 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.584726 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-iptables-alerter-script\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.585139 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.585119 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b612495-6f09-40f5-9eb9-b8e603d61d37-ovn-node-metrics-cert\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.585326 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.585280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" event={"ID":"a94ea4805b79c488e4179fc1621147fc","Type":"ContainerStarted","Data":"48c6cdda36d1feff85d819a76608b173b48187c08663ea0a2f27ead0245a5b17"} Jul 12 14:40:54.586360 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.586339 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-tmp\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.586551 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.586524 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-etc-tuned\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.586855 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.586796 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" event={"ID":"e42d2ac267c25452ab1b32df9632db77","Type":"ContainerStarted","Data":"92ea365a5c858c5a3a58e09b38740db4b54b1d7807619041c02f2f8edd1c327a"} Jul 12 14:40:54.587582 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.587528 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t2vd\" (UniqueName: \"kubernetes.io/projected/2b612495-6f09-40f5-9eb9-b8e603d61d37-kube-api-access-7t2vd\") pod \"ovnkube-node-hrksg\" (UID: \"2b612495-6f09-40f5-9eb9-b8e603d61d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.588119 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.588079 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7frr\" (UniqueName: \"kubernetes.io/projected/a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6-kube-api-access-x7frr\") pod \"tuned-9n9h2\" (UID: \"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6\") " pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.588550 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.588528 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8czt6\" (UniqueName: \"kubernetes.io/projected/3043ff23-e0c3-4c6a-80a0-80566bc79d76-kube-api-access-8czt6\") pod \"node-ca-6mbxr\" (UID: \"3043ff23-e0c3-4c6a-80a0-80566bc79d76\") " pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.594933 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.594911 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdqlz\" (UniqueName: \"kubernetes.io/projected/c1140271-ad4b-4cd7-96e8-64318c687921-kube-api-access-zdqlz\") pod \"aws-ebs-csi-driver-node-sck2x\" (UID: \"c1140271-ad4b-4cd7-96e8-64318c687921\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.595430 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.595410 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cskmg\" (UniqueName: \"kubernetes.io/projected/ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95-kube-api-access-cskmg\") pod \"iptables-alerter-r86t9\" (UID: \"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95\") " pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.596821 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.596801 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crf7v\" (UniqueName: \"kubernetes.io/projected/7aa6132f-7dbf-439b-8683-2d1aca184a98-kube-api-access-crf7v\") pod \"node-resolver-ppcrh\" (UID: \"7aa6132f-7dbf-439b-8683-2d1aca184a98\") " pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.666075 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.666043 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 14:40:54.674749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-system-cni-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.674749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674708 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:54.674749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-daemon-config\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-kubelet\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674812 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-etc-kubernetes\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674866 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-netns\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674888 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-multus-certs\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674913 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-tuning-conf-dir\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-socket-dir-parent\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-cni-bin\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.674999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-cnibin\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-cni-binary-copy\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-cni-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-os-release\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675110 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-conf-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675134 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrx7\" (UniqueName: \"kubernetes.io/projected/903d8cfd-d90e-42ca-8a0e-54b2763363ac-kube-api-access-qnrx7\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx7qw\" (UniqueName: \"kubernetes.io/projected/9ab38523-f4fb-4fb3-9111-4dd8070e71de-kube-api-access-rx7qw\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ee45648f-77ef-4944-aa18-e1c690675ef4-kubelet-config\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675241 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-cnibin\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/903d8cfd-d90e-42ca-8a0e-54b2763363ac-cni-binary-copy\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675290 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-hostroot\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675315 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-os-release\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675345 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-k8s-cni-cncf-io\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.675483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675372 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdc8f\" (UniqueName: \"kubernetes.io/projected/0e85603c-a47b-4425-b415-f66384553038-kube-api-access-fdc8f\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.676109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675396 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-cni-multus\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.676109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-system-cni-dir\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.676109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675452 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ee45648f-77ef-4944-aa18-e1c690675ef4-dbus\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.676109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.675477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.676289 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.676124 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.676289 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.676241 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-system-cni-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.676376 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.676341 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.676432 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:40:55.176397518 +0000 UTC m=+3.144280752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.676508 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-os-release\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.676667 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-conf-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677087 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-k8s-cni-cncf-io\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677100 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ee45648f-77ef-4944-aa18-e1c690675ef4-kubelet-config\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677149 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-cni-multus\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677170 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-os-release\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.677215 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.677258 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret podName:ee45648f-77ef-4944-aa18-e1c690675ef4 nodeName:}" failed. No retries permitted until 2026-07-12 14:40:55.177244525 +0000 UTC m=+3.145127759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret") pod "global-pull-secret-syncer-jbkhl" (UID: "ee45648f-77ef-4944-aa18-e1c690675ef4") : object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677327 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-system-cni-dir\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677387 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-cnibin\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677562 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ee45648f-77ef-4944-aa18-e1c690675ef4-dbus\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677615 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-multus-certs\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.678437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677810 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/903d8cfd-d90e-42ca-8a0e-54b2763363ac-cni-binary-copy\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677867 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-hostroot\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677912 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-etc-kubernetes\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.677980 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-kubelet\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678042 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-var-lib-cni-bin\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678096 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678173 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-tuning-conf-dir\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678226 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-socket-dir-parent\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678265 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-host-run-netns\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678322 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-cni-dir\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678363 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e85603c-a47b-4425-b415-f66384553038-cnibin\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678406 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/903d8cfd-d90e-42ca-8a0e-54b2763363ac-multus-daemon-config\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.679099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.678692 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e85603c-a47b-4425-b415-f66384553038-cni-binary-copy\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.682771 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.682752 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 14:40:54.682886 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.682774 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 14:40:54.682886 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.682799 2573 projected.go:196] Error preparing data for projected volume kube-api-access-25442 for pod openshift-network-diagnostics/network-check-target-2pxd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:54.682886 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:54.682856 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442 podName:1eda50bf-f2de-4174-bbd8-eef8fa991ea1 nodeName:}" failed. No retries permitted until 2026-07-12 14:40:55.182839917 +0000 UTC m=+3.150723154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-25442" (UniqueName: "kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442") pod "network-check-target-2pxd8" (UID: "1eda50bf-f2de-4174-bbd8-eef8fa991ea1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:54.684955 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.684927 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx7qw\" (UniqueName: \"kubernetes.io/projected/9ab38523-f4fb-4fb3-9111-4dd8070e71de-kube-api-access-rx7qw\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:54.685220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.685195 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdc8f\" (UniqueName: \"kubernetes.io/projected/0e85603c-a47b-4425-b415-f66384553038-kube-api-access-fdc8f\") pod \"multus-additional-cni-plugins-567hq\" (UID: \"0e85603c-a47b-4425-b415-f66384553038\") " pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.685851 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.685832 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrx7\" (UniqueName: \"kubernetes.io/projected/903d8cfd-d90e-42ca-8a0e-54b2763363ac-kube-api-access-qnrx7\") pod \"multus-rkmbl\" (UID: \"903d8cfd-d90e-42ca-8a0e-54b2763363ac\") " pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.750072 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.750037 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" Jul 12 14:40:54.760281 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.760254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ppcrh" Jul 12 14:40:54.761243 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.761211 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1140271_ad4b_4cd7_96e8_64318c687921.slice/crio-b3746753c95d0af13b344d55d783cc8b2bc427ac3ac6bbdf72ea1026921b21ff WatchSource:0}: Error finding container b3746753c95d0af13b344d55d783cc8b2bc427ac3ac6bbdf72ea1026921b21ff: Status 404 returned error can't find the container with id b3746753c95d0af13b344d55d783cc8b2bc427ac3ac6bbdf72ea1026921b21ff Jul 12 14:40:54.769441 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.769413 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa6132f_7dbf_439b_8683_2d1aca184a98.slice/crio-59ad9d8de07bc23ec8842360d40a371aa58efc496902ed41e2b7837a031f0b2b WatchSource:0}: Error finding container 59ad9d8de07bc23ec8842360d40a371aa58efc496902ed41e2b7837a031f0b2b: Status 404 returned error can't find the container with id 59ad9d8de07bc23ec8842360d40a371aa58efc496902ed41e2b7837a031f0b2b Jul 12 14:40:54.769979 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.769875 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r86t9" Jul 12 14:40:54.778646 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.778625 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:40:54.779285 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.779252 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfe44e8_b1a0_4ea2_82bf_f2eca4deed95.slice/crio-30b7922318c70f5d17b356d26711135cc5a6098928e5acb8a3ae45ce9889a55f WatchSource:0}: Error finding container 30b7922318c70f5d17b356d26711135cc5a6098928e5acb8a3ae45ce9889a55f: Status 404 returned error can't find the container with id 30b7922318c70f5d17b356d26711135cc5a6098928e5acb8a3ae45ce9889a55f Jul 12 14:40:54.785956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.785769 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:40:54.787318 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.787295 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b612495_6f09_40f5_9eb9_b8e603d61d37.slice/crio-0e3420ea30096818a2f4fd4ce14c754349b0b16d832b601859bc76ab65264ebf WatchSource:0}: Error finding container 0e3420ea30096818a2f4fd4ce14c754349b0b16d832b601859bc76ab65264ebf: Status 404 returned error can't find the container with id 0e3420ea30096818a2f4fd4ce14c754349b0b16d832b601859bc76ab65264ebf Jul 12 14:40:54.793780 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.793763 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" Jul 12 14:40:54.795398 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.795376 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63be01d_18d1_40e1_8864_fb315b971072.slice/crio-f549e9ebb73d8aa7732306c258fd873ea7faf83a45314b936bbf29aee5e84b70 WatchSource:0}: Error finding container f549e9ebb73d8aa7732306c258fd873ea7faf83a45314b936bbf29aee5e84b70: Status 404 returned error can't find the container with id f549e9ebb73d8aa7732306c258fd873ea7faf83a45314b936bbf29aee5e84b70 Jul 12 14:40:54.800905 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.800883 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6mbxr" Jul 12 14:40:54.803360 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.803340 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45a7db2_d3ec_4389_bb88_c1b78fe6e9b6.slice/crio-dd0a9202562299fd9c2521b9c6b591cf4233b1c50a52da50ccbb324d74e93cf9 WatchSource:0}: Error finding container dd0a9202562299fd9c2521b9c6b591cf4233b1c50a52da50ccbb324d74e93cf9: Status 404 returned error can't find the container with id dd0a9202562299fd9c2521b9c6b591cf4233b1c50a52da50ccbb324d74e93cf9 Jul 12 14:40:54.809382 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.809360 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-567hq" Jul 12 14:40:54.811720 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.811682 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3043ff23_e0c3_4c6a_80a0_80566bc79d76.slice/crio-86c0fb4c796b6f0fb64c297f5a3f17900a6e665a096a37da2f64b0cef1d9e746 WatchSource:0}: Error finding container 86c0fb4c796b6f0fb64c297f5a3f17900a6e665a096a37da2f64b0cef1d9e746: Status 404 returned error can't find the container with id 86c0fb4c796b6f0fb64c297f5a3f17900a6e665a096a37da2f64b0cef1d9e746 Jul 12 14:40:54.815797 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:54.815778 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rkmbl" Jul 12 14:40:54.821228 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.821202 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e85603c_a47b_4425_b415_f66384553038.slice/crio-8d1613fa0ae39d7d298702682566fcb41dee8c78333f53ad992b507b53443388 WatchSource:0}: Error finding container 8d1613fa0ae39d7d298702682566fcb41dee8c78333f53ad992b507b53443388: Status 404 returned error can't find the container with id 8d1613fa0ae39d7d298702682566fcb41dee8c78333f53ad992b507b53443388 Jul 12 14:40:54.830402 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:40:54.830376 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod903d8cfd_d90e_42ca_8a0e_54b2763363ac.slice/crio-04c85736f7adbe8857e7ecc96b024b02a06f95be28b4e245fa4db9f7c58ca1ab WatchSource:0}: Error finding container 04c85736f7adbe8857e7ecc96b024b02a06f95be28b4e245fa4db9f7c58ca1ab: Status 404 returned error can't find the container with id 04c85736f7adbe8857e7ecc96b024b02a06f95be28b4e245fa4db9f7c58ca1ab Jul 12 14:40:55.178359 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.178329 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:55.178503 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.178373 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:55.178557 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.178510 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:55.178598 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.178586 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:55.178638 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.178592 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret podName:ee45648f-77ef-4944-aa18-e1c690675ef4 nodeName:}" failed. No retries permitted until 2026-07-12 14:40:56.178572405 +0000 UTC m=+4.146455638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret") pod "global-pull-secret-syncer-jbkhl" (UID: "ee45648f-77ef-4944-aa18-e1c690675ef4") : object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:55.178680 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.178648 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:40:56.178633923 +0000 UTC m=+4.146517162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:55.279403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.279373 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:55.279575 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.279525 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 14:40:55.279575 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.279550 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 14:40:55.279575 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.279561 2573 projected.go:196] Error preparing data for projected volume kube-api-access-25442 for pod openshift-network-diagnostics/network-check-target-2pxd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:55.279710 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:55.279617 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442 podName:1eda50bf-f2de-4174-bbd8-eef8fa991ea1 nodeName:}" failed. No retries permitted until 2026-07-12 14:40:56.27959828 +0000 UTC m=+4.247481530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-25442" (UniqueName: "kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442") pod "network-check-target-2pxd8" (UID: "1eda50bf-f2de-4174-bbd8-eef8fa991ea1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:55.508250 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.508148 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 14:35:54 +0000 UTC" deadline="2028-01-31 01:20:35.536467559 +0000 UTC" Jul 12 14:40:55.508250 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.508195 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13618h39m40.028276288s" Jul 12 14:40:55.589623 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.589588 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ppcrh" event={"ID":"7aa6132f-7dbf-439b-8683-2d1aca184a98","Type":"ContainerStarted","Data":"59ad9d8de07bc23ec8842360d40a371aa58efc496902ed41e2b7837a031f0b2b"} Jul 12 14:40:55.590967 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.590935 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerStarted","Data":"8d1613fa0ae39d7d298702682566fcb41dee8c78333f53ad992b507b53443388"} Jul 12 14:40:55.592327 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.592298 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6mbxr" event={"ID":"3043ff23-e0c3-4c6a-80a0-80566bc79d76","Type":"ContainerStarted","Data":"86c0fb4c796b6f0fb64c297f5a3f17900a6e665a096a37da2f64b0cef1d9e746"} Jul 12 14:40:55.593700 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.593675 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" event={"ID":"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6","Type":"ContainerStarted","Data":"dd0a9202562299fd9c2521b9c6b591cf4233b1c50a52da50ccbb324d74e93cf9"} Jul 12 14:40:55.594892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.594867 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2pmbg" event={"ID":"a63be01d-18d1-40e1-8864-fb315b971072","Type":"ContainerStarted","Data":"f549e9ebb73d8aa7732306c258fd873ea7faf83a45314b936bbf29aee5e84b70"} Jul 12 14:40:55.596067 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.596042 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r86t9" event={"ID":"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95","Type":"ContainerStarted","Data":"30b7922318c70f5d17b356d26711135cc5a6098928e5acb8a3ae45ce9889a55f"} Jul 12 14:40:55.597437 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.597381 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" event={"ID":"c1140271-ad4b-4cd7-96e8-64318c687921","Type":"ContainerStarted","Data":"b3746753c95d0af13b344d55d783cc8b2bc427ac3ac6bbdf72ea1026921b21ff"} Jul 12 14:40:55.598790 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.598766 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rkmbl" event={"ID":"903d8cfd-d90e-42ca-8a0e-54b2763363ac","Type":"ContainerStarted","Data":"04c85736f7adbe8857e7ecc96b024b02a06f95be28b4e245fa4db9f7c58ca1ab"} Jul 12 14:40:55.600126 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:55.600064 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"0e3420ea30096818a2f4fd4ce14c754349b0b16d832b601859bc76ab65264ebf"} Jul 12 14:40:56.190424 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.189426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:56.190424 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.189556 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:56.190424 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.189780 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:56.190424 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.189819 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:40:58.189798522 +0000 UTC m=+6.157681768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:56.190424 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.189871 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:56.190424 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.189922 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret podName:ee45648f-77ef-4944-aa18-e1c690675ef4 nodeName:}" failed. No retries permitted until 2026-07-12 14:40:58.189910756 +0000 UTC m=+6.157793991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret") pod "global-pull-secret-syncer-jbkhl" (UID: "ee45648f-77ef-4944-aa18-e1c690675ef4") : object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:56.290488 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.290367 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:56.290634 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.290540 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 14:40:56.290634 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.290569 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 14:40:56.290634 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.290581 2573 projected.go:196] Error preparing data for projected volume kube-api-access-25442 for pod openshift-network-diagnostics/network-check-target-2pxd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:56.290794 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.290655 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442 podName:1eda50bf-f2de-4174-bbd8-eef8fa991ea1 nodeName:}" failed. No retries permitted until 2026-07-12 14:40:58.290634763 +0000 UTC m=+6.258518001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-25442" (UniqueName: "kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442") pod "network-check-target-2pxd8" (UID: "1eda50bf-f2de-4174-bbd8-eef8fa991ea1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:56.508697 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.508637 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 14:35:54 +0000 UTC" deadline="2028-02-29 03:29:56.690532917 +0000 UTC" Jul 12 14:40:56.508697 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.508696 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14316h49m0.18184185s" Jul 12 14:40:56.570669 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.570641 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:56.570799 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.570772 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:40:56.571218 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.571199 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:56.571330 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.571310 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:40:56.571409 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.571386 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:56.571499 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:56.571481 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:40:56.626738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:56.626648 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" event={"ID":"e42d2ac267c25452ab1b32df9632db77","Type":"ContainerStarted","Data":"f341f33c916a26b1ae082d662e29cb3dae30dcb44716585b7b7ef2afe0343657"} Jul 12 14:40:57.647328 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:57.647241 2573 generic.go:356] "Generic (PLEG): container finished" podID="a94ea4805b79c488e4179fc1621147fc" containerID="53cf3cac0305a7ae84a947c89c7c934dfc75dc1397c683cbc2fe55f8cdbf6b5a" exitCode=0 Jul 12 14:40:57.647792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:57.647434 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" event={"ID":"a94ea4805b79c488e4179fc1621147fc","Type":"ContainerDied","Data":"53cf3cac0305a7ae84a947c89c7c934dfc75dc1397c683cbc2fe55f8cdbf6b5a"} Jul 12 14:40:57.665302 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:57.665228 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-78.ec2.internal" podStartSLOduration=4.665204049 podStartE2EDuration="4.665204049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:40:56.643208335 +0000 UTC m=+4.611091589" watchObservedRunningTime="2026-07-12 14:40:57.665204049 +0000 UTC m=+5.633087302" Jul 12 14:40:58.210724 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:58.210680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:58.210893 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:58.210747 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:58.210948 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.210900 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:58.211003 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.210961 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:41:02.210942247 +0000 UTC m=+10.178825490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:40:58.211103 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.211051 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:58.211103 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.211089 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret podName:ee45648f-77ef-4944-aa18-e1c690675ef4 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:02.211077666 +0000 UTC m=+10.178960902 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret") pod "global-pull-secret-syncer-jbkhl" (UID: "ee45648f-77ef-4944-aa18-e1c690675ef4") : object "kube-system"/"original-pull-secret" not registered Jul 12 14:40:58.311556 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:58.311487 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:58.311750 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.311683 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 14:40:58.311750 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.311705 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 14:40:58.311750 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.311717 2573 projected.go:196] Error preparing data for projected volume kube-api-access-25442 for pod openshift-network-diagnostics/network-check-target-2pxd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:58.311910 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.311774 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442 podName:1eda50bf-f2de-4174-bbd8-eef8fa991ea1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:02.311755916 +0000 UTC m=+10.279639155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-25442" (UniqueName: "kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442") pod "network-check-target-2pxd8" (UID: "1eda50bf-f2de-4174-bbd8-eef8fa991ea1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:40:58.570673 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:58.570592 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:40:58.570827 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:58.570604 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:40:58.570827 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:40:58.570741 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:40:58.570827 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.570734 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:40:58.571000 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.570846 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:40:58.571000 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:40:58.570915 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:00.570162 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:00.570072 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:00.570616 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:00.570071 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:00.570616 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:00.570256 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:00.570616 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:00.570211 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:00.571212 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:00.570752 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:00.571212 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:00.571148 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:02.246338 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:02.246301 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:02.246866 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:02.246366 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:02.246866 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.246515 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:41:02.246866 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.246575 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:41:10.246557301 +0000 UTC m=+18.214440537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:41:02.247084 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.246946 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 14:41:02.247084 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.247005 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret podName:ee45648f-77ef-4944-aa18-e1c690675ef4 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:10.246989334 +0000 UTC m=+18.214872569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret") pod "global-pull-secret-syncer-jbkhl" (UID: "ee45648f-77ef-4944-aa18-e1c690675ef4") : object "kube-system"/"original-pull-secret" not registered Jul 12 14:41:02.347463 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:02.347280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:02.347643 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.347498 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 14:41:02.347643 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.347517 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 14:41:02.347643 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.347528 2573 projected.go:196] Error preparing data for projected volume kube-api-access-25442 for pod openshift-network-diagnostics/network-check-target-2pxd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:41:02.347643 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.347585 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442 podName:1eda50bf-f2de-4174-bbd8-eef8fa991ea1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:10.347568781 +0000 UTC m=+18.315452012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-25442" (UniqueName: "kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442") pod "network-check-target-2pxd8" (UID: "1eda50bf-f2de-4174-bbd8-eef8fa991ea1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:41:02.571057 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:02.570952 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:02.571269 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:02.571078 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:02.571269 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.571095 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:02.571269 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.571179 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:02.571269 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:02.571243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:02.571487 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:02.571319 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:04.570747 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:04.570713 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:04.571250 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:04.570713 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:04.571250 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:04.570851 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:04.571250 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:04.570948 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:04.571250 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:04.570726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:04.571250 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:04.571059 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:06.570284 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:06.570245 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:06.570284 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:06.570280 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:06.570766 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:06.570246 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:06.570766 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:06.570386 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:06.570766 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:06.570483 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:06.570766 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:06.570562 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:08.570468 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:08.570432 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:08.570942 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:08.570442 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:08.570942 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:08.570568 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:08.570942 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:08.570667 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:08.570942 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:08.570706 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:08.570942 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:08.570823 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:10.305690 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:10.305646 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:10.306205 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:10.305739 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:10.306205 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.305818 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:41:10.306205 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.305842 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 14:41:10.306205 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.305895 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret podName:ee45648f-77ef-4944-aa18-e1c690675ef4 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:26.305875568 +0000 UTC m=+34.273758800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret") pod "global-pull-secret-syncer-jbkhl" (UID: "ee45648f-77ef-4944-aa18-e1c690675ef4") : object "kube-system"/"original-pull-secret" not registered Jul 12 14:41:10.306205 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.305915 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:41:26.30590531 +0000 UTC m=+34.273788545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:41:10.406549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:10.406509 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:10.406736 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.406677 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 14:41:10.406736 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.406702 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 14:41:10.406736 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.406715 2573 projected.go:196] Error preparing data for projected volume kube-api-access-25442 for pod openshift-network-diagnostics/network-check-target-2pxd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:41:10.406897 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.406776 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442 podName:1eda50bf-f2de-4174-bbd8-eef8fa991ea1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:26.406758578 +0000 UTC m=+34.374641814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-25442" (UniqueName: "kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442") pod "network-check-target-2pxd8" (UID: "1eda50bf-f2de-4174-bbd8-eef8fa991ea1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:41:10.570708 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:10.570628 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:10.570878 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:10.570629 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:10.570878 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.570747 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:10.570878 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:10.570764 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:10.570878 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.570865 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:10.571108 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:10.570971 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:12.570733 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:12.570700 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:12.571197 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:12.570800 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:12.571197 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:12.570877 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:12.571197 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:12.570986 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:12.571197 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:12.571058 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:12.571197 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:12.571154 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:13.685534 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.685229 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rkmbl" event={"ID":"903d8cfd-d90e-42ca-8a0e-54b2763363ac","Type":"ContainerStarted","Data":"0d662ac651f1758c2465413f45c3a9b126889fb56e3a4c80dda6d2ed56fd18e4"} Jul 12 14:41:13.686901 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.686875 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ppcrh" event={"ID":"7aa6132f-7dbf-439b-8683-2d1aca184a98","Type":"ContainerStarted","Data":"cfbc8614090bd9027a213ca2e6d2640519d41afe45e695126229bbf8193216a0"} Jul 12 14:41:13.688617 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.688589 2573 generic.go:356] "Generic (PLEG): container finished" podID="0e85603c-a47b-4425-b415-f66384553038" containerID="bf06bdc82c4fa5f4b3386c19a42c8396fe8df0c41a93fa14e915e398edb76b32" exitCode=0 Jul 12 14:41:13.688712 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.688655 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerDied","Data":"bf06bdc82c4fa5f4b3386c19a42c8396fe8df0c41a93fa14e915e398edb76b32"} Jul 12 14:41:13.690190 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.690150 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6mbxr" event={"ID":"3043ff23-e0c3-4c6a-80a0-80566bc79d76","Type":"ContainerStarted","Data":"178144376fb139e8ce96cd59845b74d90619c7c629242a50baf75b25df3990bb"} Jul 12 14:41:13.691598 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.691567 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" event={"ID":"a45a7db2-d3ec-4389-bb88-c1b78fe6e9b6","Type":"ContainerStarted","Data":"d09622512e04574e62d2b4a94d6490962c37563dfd1c146721dfd2cc79f94cf2"} Jul 12 14:41:13.693065 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.693043 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2pmbg" event={"ID":"a63be01d-18d1-40e1-8864-fb315b971072","Type":"ContainerStarted","Data":"a4c21d009f42fe2efdb1fd3d28e9212075d8deb4a9ac5ea6775c820edf904055"} Jul 12 14:41:13.694411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.694391 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" event={"ID":"c1140271-ad4b-4cd7-96e8-64318c687921","Type":"ContainerStarted","Data":"4fa96fba63f535d3fd62813582a0bc849167a7d239d0035a0d497913c4e45985"} Jul 12 14:41:13.696244 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.696220 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" event={"ID":"a94ea4805b79c488e4179fc1621147fc","Type":"ContainerStarted","Data":"07711d3666055e5e0e0b5702681e31a0e1fcd5bb289994aa1f044f2671f6c05e"} Jul 12 14:41:13.702892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.701814 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rkmbl" podStartSLOduration=3.195609331 podStartE2EDuration="21.701798757s" totalImagesPullingTime="18.506189426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:13.701308696 +0000 UTC m=+21.669191948" watchObservedRunningTime="2026-07-12 14:41:13.701798757 +0000 UTC m=+21.669682010" Jul 12 14:41:13.733378 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.733303 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-78.ec2.internal" podStartSLOduration=20.733280853 podStartE2EDuration="20.733280853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:13.732626227 +0000 UTC m=+21.700509490" watchObservedRunningTime="2026-07-12 14:41:13.733280853 +0000 UTC m=+21.701164106" Jul 12 14:41:13.749485 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.749442 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ppcrh" podStartSLOduration=7.512510773 podStartE2EDuration="21.749422281s" totalImagesPullingTime="14.236911508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:13.749328893 +0000 UTC m=+21.717212147" watchObservedRunningTime="2026-07-12 14:41:13.749422281 +0000 UTC m=+21.717305534" Jul 12 14:41:13.766457 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.766413 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9n9h2" podStartSLOduration=3.270675254 podStartE2EDuration="21.76639606s" totalImagesPullingTime="18.495720806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:13.76631035 +0000 UTC m=+21.734193603" watchObservedRunningTime="2026-07-12 14:41:13.76639606 +0000 UTC m=+21.734279313" Jul 12 14:41:13.781466 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.781416 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6mbxr" podStartSLOduration=3.306066922 podStartE2EDuration="21.781399449s" totalImagesPullingTime="18.475332527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:13.781294296 +0000 UTC m=+21.749177548" watchObservedRunningTime="2026-07-12 14:41:13.781399449 +0000 UTC m=+21.749282698" Jul 12 14:41:13.808691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.808647 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2pmbg" podStartSLOduration=3.3046299230000002 podStartE2EDuration="21.808630598s" totalImagesPullingTime="18.504000675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:13.808536122 +0000 UTC m=+21.776419374" watchObservedRunningTime="2026-07-12 14:41:13.808630598 +0000 UTC m=+21.776513921" Jul 12 14:41:13.981884 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.981715 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:41:13.982415 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:13.982393 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:41:14.570798 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.570767 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:14.570798 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.570804 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:14.571052 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.570771 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:14.571052 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:14.570897 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:14.571052 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:14.570998 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:14.571211 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:14.571102 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:14.703856 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.703829 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:41:14.704265 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.704174 2573 generic.go:356] "Generic (PLEG): container finished" podID="2b612495-6f09-40f5-9eb9-b8e603d61d37" containerID="d546a6a938c61efcaf1732b68b0bf3bfbd2973fb3845a6429e25b225870f8e00" exitCode=1 Jul 12 14:41:14.704325 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.704279 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"8341ee17d0f56c0dfc35693c99b2161df8e1382b2a9ed3c397ce9988ca674628"} Jul 12 14:41:14.704378 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.704359 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"20230f31486020fdacb560e03efdf214e4ed24cd0aecaf405b5747b15b2c9f18"} Jul 12 14:41:14.704427 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.704391 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"80852b2edd68d40fcfd7fa12660d470a1f85513ba01673bb3cd8e62525f9f66b"} Jul 12 14:41:14.704427 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.704422 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"3b41a579999e1188dd0520205710e9cb3001fa9ab8bea07c9435d3da814d03fd"} Jul 12 14:41:14.704524 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.704467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerDied","Data":"d546a6a938c61efcaf1732b68b0bf3bfbd2973fb3845a6429e25b225870f8e00"} Jul 12 14:41:14.704524 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.704518 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"d08c6118e25937b9fae0661fcadc7a59b8a63a2dae6ea7e3ce93c82a12e66a3e"} Jul 12 14:41:14.929620 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:14.929460 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 14:41:15.552193 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:15.552080 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T14:41:14.929603829Z","UUID":"88464853-9e8e-4bcb-8a93-ab5bddd83f25","Handler":null,"Name":"","Endpoint":""} Jul 12 14:41:15.553907 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:15.553886 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 14:41:15.554048 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:15.553925 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 14:41:15.708389 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:15.708354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r86t9" event={"ID":"ddfe44e8-b1a0-4ea2-82bf-f2eca4deed95","Type":"ContainerStarted","Data":"c3864efc658efd79417a37ee1ff42ab350029bf75b0bfe44f35dd6ad7a49e15d"} Jul 12 14:41:15.710654 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:15.710630 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 14:41:15.710654 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:15.710638 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" event={"ID":"c1140271-ad4b-4cd7-96e8-64318c687921","Type":"ContainerStarted","Data":"143176eb9d1ec714d743ea0984b12350c56bcf727613bf2b75a4a57592180c02"} Jul 12 14:41:15.722542 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:15.722497 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r86t9" podStartSLOduration=5.202366974 podStartE2EDuration="23.722481949s" totalImagesPullingTime="18.520114975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:15.722026369 +0000 UTC m=+23.689909614" watchObservedRunningTime="2026-07-12 14:41:15.722481949 +0000 UTC m=+23.690365248" Jul 12 14:41:16.570314 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:16.570238 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:16.570580 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:16.570238 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:16.570580 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:16.570371 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:16.570580 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:16.570367 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:16.570580 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:16.570480 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:16.570580 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:16.570568 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:16.715234 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:16.715197 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" event={"ID":"c1140271-ad4b-4cd7-96e8-64318c687921","Type":"ContainerStarted","Data":"be962773ff683ebc0b9ebc2a680177bd58dc409e59699475a1faeb144cfa6f9a"} Jul 12 14:41:16.731490 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:16.731440 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sck2x" podStartSLOduration=3.630862965 podStartE2EDuration="24.731423148s" totalImagesPullingTime="21.100560183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:16.730820124 +0000 UTC m=+24.698703375" watchObservedRunningTime="2026-07-12 14:41:16.731423148 +0000 UTC m=+24.699306400" Jul 12 14:41:18.570702 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.570665 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:18.571312 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.570664 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:18.571312 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.570796 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:18.571312 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:18.570796 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:18.571312 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:18.570896 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:18.571312 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:18.570964 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:18.722491 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.722415 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:41:18.722773 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.722753 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"31f595b48821c1215ee20ecc6d5b6ce00bbf645e7f59016515fcca25c6d6407f"} Jul 12 14:41:18.724444 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.724423 2573 generic.go:356] "Generic (PLEG): container finished" podID="0e85603c-a47b-4425-b415-f66384553038" containerID="6211da51136bcad7b076f60330950c03b0b61d3aca72a7f39b9bff918db51572" exitCode=0 Jul 12 14:41:18.724565 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.724465 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerDied","Data":"6211da51136bcad7b076f60330950c03b0b61d3aca72a7f39b9bff918db51572"} Jul 12 14:41:18.860155 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.860115 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:41:18.860319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.860259 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 14:41:18.861105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:18.861078 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2pmbg" Jul 12 14:41:19.728910 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:19.728878 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerStarted","Data":"0725a60030bcac4766ad9f7dcba8dfb8b1bd98f026d2c7a7ba5deac6905cefcc"} Jul 12 14:41:20.570659 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.570390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:20.570659 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.570405 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:20.570837 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:20.570664 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:20.570837 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.570391 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:20.570837 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:20.570774 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:20.570837 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:20.570788 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:20.734050 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.734023 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:41:20.734564 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.734401 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"b0ab25e9b9532149bdd4f3bdc1994e35b61fab1a775d2029f087165da6ccb785"} Jul 12 14:41:20.734645 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.734628 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:41:20.734863 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.734843 2573 scope.go:117] "RemoveContainer" containerID="d546a6a938c61efcaf1732b68b0bf3bfbd2973fb3845a6429e25b225870f8e00" Jul 12 14:41:20.736891 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.736863 2573 generic.go:356] "Generic (PLEG): container finished" podID="0e85603c-a47b-4425-b415-f66384553038" containerID="0725a60030bcac4766ad9f7dcba8dfb8b1bd98f026d2c7a7ba5deac6905cefcc" exitCode=0 Jul 12 14:41:20.736992 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.736899 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerDied","Data":"0725a60030bcac4766ad9f7dcba8dfb8b1bd98f026d2c7a7ba5deac6905cefcc"} Jul 12 14:41:20.750902 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:20.750882 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:41:21.742745 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.742669 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:41:21.743141 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.743051 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" event={"ID":"2b612495-6f09-40f5-9eb9-b8e603d61d37","Type":"ContainerStarted","Data":"b9e14c59c0c5bfdfea15dd4aaa69abf2d09cf9f0ba6875a37a241d6f5781aa0a"} Jul 12 14:41:21.743279 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.743254 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:41:21.743428 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.743412 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:41:21.745524 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.745504 2573 generic.go:356] "Generic (PLEG): container finished" podID="0e85603c-a47b-4425-b415-f66384553038" containerID="7c9b2b7f24d83543ce3d937c8f64f3f8bcf4d3b357917944e67e6162d4487494" exitCode=0 Jul 12 14:41:21.745639 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.745539 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerDied","Data":"7c9b2b7f24d83543ce3d937c8f64f3f8bcf4d3b357917944e67e6162d4487494"} Jul 12 14:41:21.758237 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.758217 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:41:21.774336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:21.774286 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" podStartSLOduration=10.986253687 podStartE2EDuration="29.774271887s" totalImagesPullingTime="18.7880182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:21.772990607 +0000 UTC m=+29.740873861" watchObservedRunningTime="2026-07-12 14:41:21.774271887 +0000 UTC m=+29.742155169" Jul 12 14:41:22.578415 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:22.578385 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbkhl"] Jul 12 14:41:22.578559 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:22.578527 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2pxd8"] Jul 12 14:41:22.578965 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:22.578734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:22.578965 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:22.578860 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:22.578965 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:22.578898 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjw4t"] Jul 12 14:41:22.579291 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:22.579259 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:22.579466 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:22.579438 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:22.580996 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:22.580973 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:22.581145 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:22.581127 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:22.747899 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:22.747818 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:22.748366 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:22.747920 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:24.573692 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:24.573662 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:24.574344 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:24.573758 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:24.574344 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:24.573778 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:24.574344 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:24.573854 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:24.574344 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:24.573892 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:24.574344 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:24.573957 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:26.326075 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:26.326039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:26.326503 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:26.326102 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:26.326503 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.326173 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 14:41:26.326503 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.326216 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:41:26.326503 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.326232 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret podName:ee45648f-77ef-4944-aa18-e1c690675ef4 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:58.326216756 +0000 UTC m=+66.294099987 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret") pod "global-pull-secret-syncer-jbkhl" (UID: "ee45648f-77ef-4944-aa18-e1c690675ef4") : object "kube-system"/"original-pull-secret" not registered Jul 12 14:41:26.326503 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.326271 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:41:58.326254686 +0000 UTC m=+66.294137920 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 14:41:26.427186 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:26.427148 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:26.427333 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.427313 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 14:41:26.427380 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.427340 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 14:41:26.427380 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.427353 2573 projected.go:196] Error preparing data for projected volume kube-api-access-25442 for pod openshift-network-diagnostics/network-check-target-2pxd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:41:26.427471 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.427409 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442 podName:1eda50bf-f2de-4174-bbd8-eef8fa991ea1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:58.427391874 +0000 UTC m=+66.395275113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-25442" (UniqueName: "kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442") pod "network-check-target-2pxd8" (UID: "1eda50bf-f2de-4174-bbd8-eef8fa991ea1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 14:41:26.570753 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:26.570710 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:26.570947 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:26.570710 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:26.570947 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:26.570835 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:26.570947 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.570829 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:41:26.571131 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.570967 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbkhl" podUID="ee45648f-77ef-4944-aa18-e1c690675ef4" Jul 12 14:41:26.571131 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:26.571058 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2pxd8" podUID="1eda50bf-f2de-4174-bbd8-eef8fa991ea1" Jul 12 14:41:27.340355 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.340181 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-78.ec2.internal" event="NodeReady" Jul 12 14:41:27.340741 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.340475 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 14:41:27.372198 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.372164 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-pstf2"] Jul 12 14:41:27.383719 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.383696 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-765998dfdb-wtmkw"] Jul 12 14:41:27.383833 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.383821 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:27.386121 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.386097 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 12 14:41:27.386121 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.386114 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8rjvj\"" Jul 12 14:41:27.386467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.386448 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 12 14:41:27.396073 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.396051 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-pstf2"] Jul 12 14:41:27.396155 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.396091 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4rjb5"] Jul 12 14:41:27.396155 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.396102 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.398051 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.398028 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 14:41:27.398137 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.398059 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 14:41:27.398137 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.398040 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zmjmj\"" Jul 12 14:41:27.398211 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.398152 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 14:41:27.405041 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.404299 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 14:41:27.408262 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.408242 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ckf4h"] Jul 12 14:41:27.408397 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.408380 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.410301 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.410284 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 14:41:27.410411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.410317 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 14:41:27.410523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.410495 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ff579\"" Jul 12 14:41:27.420690 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.420674 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-765998dfdb-wtmkw"] Jul 12 14:41:27.420771 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.420700 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rjb5"] Jul 12 14:41:27.420771 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.420717 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ckf4h"] Jul 12 14:41:27.420771 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.420753 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.422676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.422640 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 14:41:27.422761 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.422687 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 14:41:27.422811 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.422790 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 14:41:27.422850 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.422831 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6rsjr\"" Jul 12 14:41:27.536952 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.536914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1e45d8b-95d2-4193-96df-9749957d03a7-config-volume\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.536952 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.536954 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.537203 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537004 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-certificates\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537203 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537061 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-installation-pull-secrets\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537203 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537083 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4aa18e4-0242-415b-90bc-af206b09567f-cert\") pod \"ingress-canary-ckf4h\" (UID: \"b4aa18e4-0242-415b-90bc-af206b09567f\") " pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.537203 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537101 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-ca-trust-extracted\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537203 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537118 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-trusted-ca\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537203 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537162 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10e3433a-1692-4ea9-8e91-be8d0739b2d6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:27.537381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537213 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-image-registry-private-configuration\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537230 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-bound-sa-token\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537248 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:27.537381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537262 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1e45d8b-95d2-4193-96df-9749957d03a7-tmp-dir\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.537381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537275 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwspv\" (UniqueName: \"kubernetes.io/projected/b4aa18e4-0242-415b-90bc-af206b09567f-kube-api-access-gwspv\") pod \"ingress-canary-ckf4h\" (UID: \"b4aa18e4-0242-415b-90bc-af206b09567f\") " pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.537381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537307 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9fx8\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-kube-api-access-q9fx8\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537336 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.537579 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.537380 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx6d4\" (UniqueName: \"kubernetes.io/projected/d1e45d8b-95d2-4193-96df-9749957d03a7-kube-api-access-jx6d4\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.638463 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638435 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10e3433a-1692-4ea9-8e91-be8d0739b2d6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638487 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-image-registry-private-configuration\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-bound-sa-token\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1e45d8b-95d2-4193-96df-9749957d03a7-tmp-dir\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwspv\" (UniqueName: \"kubernetes.io/projected/b4aa18e4-0242-415b-90bc-af206b09567f-kube-api-access-gwspv\") pod \"ingress-canary-ckf4h\" (UID: \"b4aa18e4-0242-415b-90bc-af206b09567f\") " pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638579 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9fx8\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-kube-api-access-q9fx8\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.638660 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638646 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx6d4\" (UniqueName: \"kubernetes.io/projected/d1e45d8b-95d2-4193-96df-9749957d03a7-kube-api-access-jx6d4\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638689 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1e45d8b-95d2-4193-96df-9749957d03a7-config-volume\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:27.638723 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638748 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-certificates\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-installation-pull-secrets\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:27.638792 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:27.638821 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:27.638798 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:28.138774931 +0000 UTC m=+36.106658166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:27.638905 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:28.138886454 +0000 UTC m=+36.106769688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4aa18e4-0242-415b-90bc-af206b09567f-cert\") pod \"ingress-canary-ckf4h\" (UID: \"b4aa18e4-0242-415b-90bc-af206b09567f\") " pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-ca-trust-extracted\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638988 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-trusted-ca\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.639164 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:27.639140 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:41:27.639715 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:27.639183 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:28.139169874 +0000 UTC m=+36.107053106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:41:27.639715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.638932 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1e45d8b-95d2-4193-96df-9749957d03a7-tmp-dir\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.639715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.639433 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/10e3433a-1692-4ea9-8e91-be8d0739b2d6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:27.639715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.639451 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1e45d8b-95d2-4193-96df-9749957d03a7-config-volume\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.639715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.639698 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-certificates\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.639715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.639704 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-ca-trust-extracted\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.640064 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.640044 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-trusted-ca\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.642537 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.642512 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-image-registry-private-configuration\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.642537 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.642512 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-installation-pull-secrets\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.642695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.642675 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4aa18e4-0242-415b-90bc-af206b09567f-cert\") pod \"ingress-canary-ckf4h\" (UID: \"b4aa18e4-0242-415b-90bc-af206b09567f\") " pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.646747 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.646708 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwspv\" (UniqueName: \"kubernetes.io/projected/b4aa18e4-0242-415b-90bc-af206b09567f-kube-api-access-gwspv\") pod \"ingress-canary-ckf4h\" (UID: \"b4aa18e4-0242-415b-90bc-af206b09567f\") " pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.647076 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.647057 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx6d4\" (UniqueName: \"kubernetes.io/projected/d1e45d8b-95d2-4193-96df-9749957d03a7-kube-api-access-jx6d4\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:27.647611 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.647592 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-bound-sa-token\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.647956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.647927 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9fx8\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-kube-api-access-q9fx8\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:27.727981 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.727941 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ckf4h" Jul 12 14:41:27.762871 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.762843 2573 generic.go:356] "Generic (PLEG): container finished" podID="0e85603c-a47b-4425-b415-f66384553038" containerID="afb3310019b3b30fef35c018e9c0c09d348b7a14c45310a92b62220d475e6d97" exitCode=0 Jul 12 14:41:27.763063 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.762881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerDied","Data":"afb3310019b3b30fef35c018e9c0c09d348b7a14c45310a92b62220d475e6d97"} Jul 12 14:41:27.925671 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:27.925640 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ckf4h"] Jul 12 14:41:27.929965 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:41:27.929935 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4aa18e4_0242_415b_90bc_af206b09567f.slice/crio-4150c9ccdbd05bd8bed0e6009e20f0976bc9178a17a9cd69502fe1e71c6d770c WatchSource:0}: Error finding container 4150c9ccdbd05bd8bed0e6009e20f0976bc9178a17a9cd69502fe1e71c6d770c: Status 404 returned error can't find the container with id 4150c9ccdbd05bd8bed0e6009e20f0976bc9178a17a9cd69502fe1e71c6d770c Jul 12 14:41:28.144259 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.144222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:28.144438 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.144293 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:28.144438 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.144316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:28.144438 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:28.144387 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:41:28.144438 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:28.144414 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:41:28.144438 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:28.144424 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:41:28.144585 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:28.144387 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:41:28.144585 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:28.144454 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:29.144438672 +0000 UTC m=+37.112321903 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:41:28.144585 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:28.144484 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:29.144473208 +0000 UTC m=+37.112356442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:41:28.144585 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:28.144498 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:29.144493099 +0000 UTC m=+37.112376330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:41:28.570589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.570554 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:28.571107 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.570599 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:28.571107 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.570760 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:28.572995 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.572974 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 14:41:28.573138 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.572998 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 14:41:28.573548 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.573479 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 14:41:28.573548 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.573491 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 14:41:28.573709 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.573577 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6tgvw\"" Jul 12 14:41:28.573709 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.573632 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s8bwx\"" Jul 12 14:41:28.766341 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.766307 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ckf4h" event={"ID":"b4aa18e4-0242-415b-90bc-af206b09567f","Type":"ContainerStarted","Data":"4150c9ccdbd05bd8bed0e6009e20f0976bc9178a17a9cd69502fe1e71c6d770c"} Jul 12 14:41:28.769089 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.769059 2573 generic.go:356] "Generic (PLEG): container finished" podID="0e85603c-a47b-4425-b415-f66384553038" containerID="a2c2d22d1ce41fbdf50e577c40c386e8c8cefcb60e0951220d0b14b9b168d054" exitCode=0 Jul 12 14:41:28.769224 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:28.769103 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerDied","Data":"a2c2d22d1ce41fbdf50e577c40c386e8c8cefcb60e0951220d0b14b9b168d054"} Jul 12 14:41:29.153924 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:29.153726 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:29.154146 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:29.153952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:29.154146 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:29.153891 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:41:29.154146 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:29.154036 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:29.154146 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:29.154103 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:31.154076831 +0000 UTC m=+39.121960076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:41:29.154357 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:29.154171 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:41:29.154357 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:29.154221 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:31.15420535 +0000 UTC m=+39.122088585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:41:29.154357 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:29.154289 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:41:29.154357 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:29.154302 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:41:29.154357 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:29.154339 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:31.15432548 +0000 UTC m=+39.122208722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:41:29.774656 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:29.774621 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-567hq" event={"ID":"0e85603c-a47b-4425-b415-f66384553038","Type":"ContainerStarted","Data":"9f5d5c7103d8439f65c6998e892d3e27b6f716bac2b48333af7bb92d4021315a"} Jul 12 14:41:29.795700 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:29.795645 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-567hq" podStartSLOduration=8.272610237 podStartE2EDuration="37.795625105s" totalImagesPullingTime="29.523014868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:29.794450098 +0000 UTC m=+37.762333351" watchObservedRunningTime="2026-07-12 14:41:29.795625105 +0000 UTC m=+37.763508358" Jul 12 14:41:29.942508 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:29.942462 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:30.027892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:30.027807 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:31.170495 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:31.170396 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:31.170495 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:31.170451 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:31.170495 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:31.170496 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:31.171170 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:31.170562 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:41:31.171170 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:31.170606 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:41:31.171170 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:31.170611 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:41:31.171170 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:31.170628 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:41:31.171170 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:31.170645 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:35.170629378 +0000 UTC m=+43.138512613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:41:31.171170 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:31.170665 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:35.170652909 +0000 UTC m=+43.138536140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:41:31.171170 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:31.170678 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:35.170672299 +0000 UTC m=+43.138555530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:41:31.780873 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:31.780835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ckf4h" event={"ID":"b4aa18e4-0242-415b-90bc-af206b09567f","Type":"ContainerStarted","Data":"dc2cfbca8d8034dfbbe6d68ddacd1e31345a0a8223ef48a6f2f0865e67974612"} Jul 12 14:41:31.796417 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:31.796360 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ckf4h" podStartSLOduration=1.849346926 podStartE2EDuration="4.796345803s" totalImagesPullingTime="2.946998877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:41:31.795921636 +0000 UTC m=+39.763804889" watchObservedRunningTime="2026-07-12 14:41:31.796345803 +0000 UTC m=+39.764229034" Jul 12 14:41:32.120279 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:32.120241 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:32.174252 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:32.174218 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:35.205093 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:35.205054 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:35.205121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:35.205156 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:35.205206 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:35.205224 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:35.205261 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:35.205274 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:35.205277 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:43.205260042 +0000 UTC m=+51.173143292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:35.205292 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:43.205285835 +0000 UTC m=+51.173169066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:41:35.205501 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:35.205302 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:43.205297494 +0000 UTC m=+51.173180726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:41:35.995888 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:35.995852 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:36.096354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:36.096309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:36.962158 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:36.962122 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:37.056746 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:37.056710 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:38.392231 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:38.392199 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:38.484859 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:38.484829 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:38.540653 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:38.540615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:39.430713 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:39.430672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:39.482544 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:39.482508 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:40.899127 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:40.899094 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:41.783954 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:41.783919 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:41.838525 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:41.838496 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:43.269002 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:43.268916 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:43.269002 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:43.268977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:43.269002 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:43.269001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:43.269530 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:43.269075 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:41:43.269530 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:43.269090 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:41:43.269530 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:43.269100 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:41:43.269530 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:43.269110 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:41:43.269530 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:43.269137 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:59.269121379 +0000 UTC m=+67.237004610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:41:43.269530 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:43.269150 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:59.269144867 +0000 UTC m=+67.237028098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:41:43.269530 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:43.269161 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:41:59.269156309 +0000 UTC m=+67.237039540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:41:53.009081 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:53.009047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:53.111967 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:53.111916 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:53.519877 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:53.519845 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:53.621943 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:53.621904 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:53.761089 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:53.761066 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hrksg" Jul 12 14:41:53.921520 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:53.921483 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:54.020861 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:54.020826 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:54.203591 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:54.203502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:54.268581 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:54.268541 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:54.541264 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:54.541154 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:54.595193 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:54.595163 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:55.795834 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:55.795785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:41:58.386615 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.386577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:41:58.387145 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.386677 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:58.389809 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.389786 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 14:41:58.389911 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.389894 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 14:41:58.397697 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:58.397672 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 14:41:58.397794 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:58.397744 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:43:02.397728008 +0000 UTC m=+130.365611239 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : secret "metrics-daemon-secret" not found Jul 12 14:41:58.400854 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.400828 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ee45648f-77ef-4944-aa18-e1c690675ef4-original-pull-secret\") pod \"global-pull-secret-syncer-jbkhl\" (UID: \"ee45648f-77ef-4944-aa18-e1c690675ef4\") " pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:58.487439 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.487398 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:58.489664 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.489647 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 14:41:58.500873 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.500855 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 14:41:58.511924 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.511889 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25442\" (UniqueName: \"kubernetes.io/projected/1eda50bf-f2de-4174-bbd8-eef8fa991ea1-kube-api-access-25442\") pod \"network-check-target-2pxd8\" (UID: \"1eda50bf-f2de-4174-bbd8-eef8fa991ea1\") " pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:58.583581 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.583553 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbkhl" Jul 12 14:41:58.590424 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.590405 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s8bwx\"" Jul 12 14:41:58.598347 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.598320 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:41:58.725250 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.725142 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbkhl"] Jul 12 14:41:58.728028 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:41:58.727980 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee45648f_77ef_4944_aa18_e1c690675ef4.slice/crio-222c790d1d04ee03fd045c1932572188f29577595945cc58b9b107d150564c4b WatchSource:0}: Error finding container 222c790d1d04ee03fd045c1932572188f29577595945cc58b9b107d150564c4b: Status 404 returned error can't find the container with id 222c790d1d04ee03fd045c1932572188f29577595945cc58b9b107d150564c4b Jul 12 14:41:58.739150 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.739124 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2pxd8"] Jul 12 14:41:58.742607 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:41:58.742584 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eda50bf_f2de_4174_bbd8_eef8fa991ea1.slice/crio-2178814873f9328276aaacaace450a5f36cdc5484b2b801551057b161047e362 WatchSource:0}: Error finding container 2178814873f9328276aaacaace450a5f36cdc5484b2b801551057b161047e362: Status 404 returned error can't find the container with id 2178814873f9328276aaacaace450a5f36cdc5484b2b801551057b161047e362 Jul 12 14:41:58.845234 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.845195 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2pxd8" event={"ID":"1eda50bf-f2de-4174-bbd8-eef8fa991ea1","Type":"ContainerStarted","Data":"2178814873f9328276aaacaace450a5f36cdc5484b2b801551057b161047e362"} Jul 12 14:41:58.846188 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:58.846167 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbkhl" event={"ID":"ee45648f-77ef-4944-aa18-e1c690675ef4","Type":"ContainerStarted","Data":"222c790d1d04ee03fd045c1932572188f29577595945cc58b9b107d150564c4b"} Jul 12 14:41:59.294535 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:59.294500 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:41:59.294742 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:59.294564 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:41:59.294742 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:41:59.294595 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:41:59.294742 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:59.294708 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:41:59.294742 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:59.294711 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:41:59.294742 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:59.294738 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:41:59.294948 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:59.294756 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:41:59.294948 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:59.294779 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:42:31.294760798 +0000 UTC m=+99.262644043 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:41:59.294948 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:59.294794 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:42:31.29478619 +0000 UTC m=+99.262669436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:41:59.294948 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:41:59.294808 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:42:31.294799864 +0000 UTC m=+99.262683097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:42:03.860089 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:03.860045 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2pxd8" event={"ID":"1eda50bf-f2de-4174-bbd8-eef8fa991ea1","Type":"ContainerStarted","Data":"014e722ce8f4e0ab163e34b041c8b57658ce52d7e19528143bb3199ab17e3249"} Jul 12 14:42:03.860554 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:03.860166 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:42:03.861432 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:03.861409 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbkhl" event={"ID":"ee45648f-77ef-4944-aa18-e1c690675ef4","Type":"ContainerStarted","Data":"4e300642be4c2538d3f75aaf72db12606af04c7dcdbe812be12002b2679cc48b"} Jul 12 14:42:03.888184 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:03.888135 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jbkhl" podStartSLOduration=67.43072009 podStartE2EDuration="1m11.888120461s" totalImagesPullingTime="4.457400371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:42:03.887689477 +0000 UTC m=+71.855572741" watchObservedRunningTime="2026-07-12 14:42:03.888120461 +0000 UTC m=+71.856003713" Jul 12 14:42:03.888508 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:03.888471 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2pxd8" podStartSLOduration=67.448585292 podStartE2EDuration="1m11.888460841s" totalImagesPullingTime="4.439875549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:42:03.874166794 +0000 UTC m=+71.842050051" watchObservedRunningTime="2026-07-12 14:42:03.888460841 +0000 UTC m=+71.856344137" Jul 12 14:42:15.029025 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:15.028879 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:42:15.085781 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:15.085750 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:42:17.480856 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:17.480822 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:42:17.535433 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:17.535400 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:42:17.594920 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:17.594890 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:42:31.327897 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:31.327857 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:31.327932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:31.327967 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:42:31.328027 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:42:31.328102 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:42:31.328109 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:42:31.328119 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:42:31.328107 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:43:35.328091692 +0000 UTC m=+163.295974923 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:42:31.328180 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:43:35.328165288 +0000 UTC m=+163.296048520 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:42:31.328432 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:42:31.328202 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:43:35.328193512 +0000 UTC m=+163.296076743 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:42:34.865884 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:34.865854 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2pxd8" Jul 12 14:42:41.495863 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:41.495828 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:42:47.934042 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.933993 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv"] Jul 12 14:42:47.935927 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.935905 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:47.938004 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.937982 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 14:42:47.938691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.938663 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 12 14:42:47.938691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.938667 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 14:42:47.938860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.938667 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-99fzp\"" Jul 12 14:42:47.938860 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.938686 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 14:42:47.952734 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:47.952706 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv"] Jul 12 14:42:48.037630 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.037589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c19e3c2-b6bf-4e68-ad88-26284caac6a5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv\" (UID: \"2c19e3c2-b6bf-4e68-ad88-26284caac6a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:48.037630 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.037628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcfd2\" (UniqueName: \"kubernetes.io/projected/2c19e3c2-b6bf-4e68-ad88-26284caac6a5-kube-api-access-dcfd2\") pod \"managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv\" (UID: \"2c19e3c2-b6bf-4e68-ad88-26284caac6a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:48.138457 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.138410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c19e3c2-b6bf-4e68-ad88-26284caac6a5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv\" (UID: \"2c19e3c2-b6bf-4e68-ad88-26284caac6a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:48.138457 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.138456 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcfd2\" (UniqueName: \"kubernetes.io/projected/2c19e3c2-b6bf-4e68-ad88-26284caac6a5-kube-api-access-dcfd2\") pod \"managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv\" (UID: \"2c19e3c2-b6bf-4e68-ad88-26284caac6a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:48.140836 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.140816 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c19e3c2-b6bf-4e68-ad88-26284caac6a5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv\" (UID: \"2c19e3c2-b6bf-4e68-ad88-26284caac6a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:48.148164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.148145 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcfd2\" (UniqueName: \"kubernetes.io/projected/2c19e3c2-b6bf-4e68-ad88-26284caac6a5-kube-api-access-dcfd2\") pod \"managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv\" (UID: \"2c19e3c2-b6bf-4e68-ad88-26284caac6a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:48.253540 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.253434 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" Jul 12 14:42:48.369113 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.369075 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv"] Jul 12 14:42:48.372791 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:42:48.372765 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c19e3c2_b6bf_4e68_ad88_26284caac6a5.slice/crio-e977c710caa53df28d0580e2839f1a8adbc18bb132828cb3e105eae553c36de8 WatchSource:0}: Error finding container e977c710caa53df28d0580e2839f1a8adbc18bb132828cb3e105eae553c36de8: Status 404 returned error can't find the container with id e977c710caa53df28d0580e2839f1a8adbc18bb132828cb3e105eae553c36de8 Jul 12 14:42:48.971687 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:48.971642 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" event={"ID":"2c19e3c2-b6bf-4e68-ad88-26284caac6a5","Type":"ContainerStarted","Data":"e977c710caa53df28d0580e2839f1a8adbc18bb132828cb3e105eae553c36de8"} Jul 12 14:42:50.978127 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:50.978088 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" event={"ID":"2c19e3c2-b6bf-4e68-ad88-26284caac6a5","Type":"ContainerStarted","Data":"c9ef75dd251c21a705a5f8affc4011a161b583c563d403b6fc059b1a0a850389"} Jul 12 14:42:50.993995 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:42:50.993933 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" podStartSLOduration=1.918017892 podStartE2EDuration="3.993914598s" totalImagesPullingTime="2.075896706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:42:50.993818063 +0000 UTC m=+118.961701315" watchObservedRunningTime="2026-07-12 14:42:50.993914598 +0000 UTC m=+118.961797854" Jul 12 14:43:02.445274 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:02.445218 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:43:02.445753 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:02.445371 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 14:43:02.445753 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:02.445436 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:45:04.445417793 +0000 UTC m=+252.413301024 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : secret "metrics-daemon-secret" not found Jul 12 14:43:30.394183 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:30.394127 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" podUID="10e3433a-1692-4ea9-8e91-be8d0739b2d6" Jul 12 14:43:30.407372 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:30.407350 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" Jul 12 14:43:30.416539 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:30.416511 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4rjb5" podUID="d1e45d8b-95d2-4193-96df-9749957d03a7" Jul 12 14:43:31.070153 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:31.070123 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:43:31.070354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:31.070160 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:43:31.070354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:31.070283 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rjb5" Jul 12 14:43:31.593063 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:31.592992 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:43:35.386349 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:35.386307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:43:35.386349 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:35.386353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:35.386446 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:35.386457 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765998dfdb-wtmkw: secret "image-registry-tls" not found Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:35.386459 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:35.386519 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls podName:881b7e06-c94e-4d19-8f3d-da8a15f8efb1 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:37.386504419 +0000 UTC m=+285.354387650 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls") pod "image-registry-765998dfdb-wtmkw" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1") : secret "image-registry-tls" not found Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:35.386532 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:37.386526457 +0000 UTC m=+285.354409688 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:35.386545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:35.386623 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 14:43:35.386874 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:43:35.386644 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls podName:d1e45d8b-95d2-4193-96df-9749957d03a7 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:37.386638381 +0000 UTC m=+285.354521612 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls") pod "dns-default-4rjb5" (UID: "d1e45d8b-95d2-4193-96df-9749957d03a7") : secret "dns-default-metrics-tls" not found Jul 12 14:43:43.570533 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:43:43.570484 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:44:49.578569 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:44:49.578540 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:44:51.257845 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:44:51.257812 2573 generic.go:356] "Generic (PLEG): container finished" podID="2c19e3c2-b6bf-4e68-ad88-26284caac6a5" containerID="c9ef75dd251c21a705a5f8affc4011a161b583c563d403b6fc059b1a0a850389" exitCode=255 Jul 12 14:44:51.258232 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:44:51.257868 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" event={"ID":"2c19e3c2-b6bf-4e68-ad88-26284caac6a5","Type":"ContainerDied","Data":"c9ef75dd251c21a705a5f8affc4011a161b583c563d403b6fc059b1a0a850389"} Jul 12 14:44:51.263375 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:44:51.263359 2573 scope.go:117] "RemoveContainer" containerID="c9ef75dd251c21a705a5f8affc4011a161b583c563d403b6fc059b1a0a850389" Jul 12 14:44:52.262756 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:44:52.262723 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68dfd75dc6-bwxvv" event={"ID":"2c19e3c2-b6bf-4e68-ad88-26284caac6a5","Type":"ContainerStarted","Data":"455ab5b5311092f91dabf04e0f9ec3d83bf1fbfed4609b111f55eb48f8bf3957"} Jul 12 14:45:04.525371 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:04.525324 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:45:04.525800 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:04.525478 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 14:45:04.525800 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:04.525549 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs podName:9ab38523-f4fb-4fb3-9111-4dd8070e71de nodeName:}" failed. No retries permitted until 2026-07-12 14:47:06.525534228 +0000 UTC m=+374.493417460 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs") pod "network-metrics-daemon-fjw4t" (UID: "9ab38523-f4fb-4fb3-9111-4dd8070e71de") : secret "metrics-daemon-secret" not found Jul 12 14:45:22.818416 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.818324 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5"] Jul 12 14:45:22.821223 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.821207 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld"] Jul 12 14:45:22.821354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.821336 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" Jul 12 14:45:22.823123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.823104 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cstgc\"" Jul 12 14:45:22.824087 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.824071 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-gg4r2"] Jul 12 14:45:22.824200 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.824186 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:22.825955 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.825933 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 12 14:45:22.826110 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.826091 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 12 14:45:22.826196 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.826176 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-djnzn\"" Jul 12 14:45:22.826254 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.826191 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 12 14:45:22.826762 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.826743 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:22.828471 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.828452 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 12 14:45:22.828994 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.828978 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 12 14:45:22.829098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.828999 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dzs8v\"" Jul 12 14:45:22.829098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.829059 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 12 14:45:22.829098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.829081 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 12 14:45:22.832417 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.832397 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5"] Jul 12 14:45:22.833485 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.833466 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 12 14:45:22.837413 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.837382 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld"] Jul 12 14:45:22.838413 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.838391 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-gg4r2"] Jul 12 14:45:22.914044 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.913995 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5"] Jul 12 14:45:22.916907 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.916886 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:22.919115 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.919080 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 12 14:45:22.919238 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.919140 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 12 14:45:22.919489 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.919470 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-29sjw\"" Jul 12 14:45:22.919644 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.919629 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 12 14:45:22.919701 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.919660 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 12 14:45:22.932693 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.932666 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5"] Jul 12 14:45:22.954645 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.954613 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvhvk\" (UniqueName: \"kubernetes.io/projected/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-kube-api-access-wvhvk\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:22.954645 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.954647 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-serving-cert\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:22.954852 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.954666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-trusted-ca\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:22.954852 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.954752 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-config\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:22.954852 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.954780 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92dpw\" (UniqueName: \"kubernetes.io/projected/ee651883-6ddd-41e4-a610-9426f491a7a8-kube-api-access-92dpw\") pod \"network-check-source-869dcfcdcd-6vzg5\" (UID: \"ee651883-6ddd-41e4-a610-9426f491a7a8\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" Jul 12 14:45:22.954852 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.954828 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lnqq\" (UniqueName: \"kubernetes.io/projected/44027a77-f6d0-4165-8bf7-65cfe1babdd6-kube-api-access-8lnqq\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:22.955000 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:22.954863 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:23.051554 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.051523 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:23.055615 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvhvk\" (UniqueName: \"kubernetes.io/projected/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-kube-api-access-wvhvk\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.055677 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-serving-cert\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.055677 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4d3fd18c-856a-42ec-a45c-2087f975bd1f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.055752 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4d3fd18c-856a-42ec-a45c-2087f975bd1f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.055752 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h667r\" (UniqueName: \"kubernetes.io/projected/4d3fd18c-856a-42ec-a45c-2087f975bd1f-kube-api-access-h667r\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.055824 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lnqq\" (UniqueName: \"kubernetes.io/projected/44027a77-f6d0-4165-8bf7-65cfe1babdd6-kube-api-access-8lnqq\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:23.055824 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-trusted-ca\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.055886 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055861 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-config\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.055935 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92dpw\" (UniqueName: \"kubernetes.io/projected/ee651883-6ddd-41e4-a610-9426f491a7a8-kube-api-access-92dpw\") pod \"network-check-source-869dcfcdcd-6vzg5\" (UID: \"ee651883-6ddd-41e4-a610-9426f491a7a8\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" Jul 12 14:45:23.055935 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.055912 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:23.056055 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:23.056006 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 14:45:23.056111 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:23.056084 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls podName:44027a77-f6d0-4165-8bf7-65cfe1babdd6 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:23.556065028 +0000 UTC m=+271.523948260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls") pod "cluster-samples-operator-65db858894-7fwld" (UID: "44027a77-f6d0-4165-8bf7-65cfe1babdd6") : secret "samples-operator-tls" not found Jul 12 14:45:23.056603 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.056582 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-trusted-ca\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.056686 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.056583 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-config\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.058045 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.058030 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-serving-cert\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.064027 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.063989 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvhvk\" (UniqueName: \"kubernetes.io/projected/6a83e5a4-34bc-4272-b2b5-5efe836a0ef2-kube-api-access-wvhvk\") pod \"console-operator-7866497494-gg4r2\" (UID: \"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2\") " pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.064027 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.063996 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92dpw\" (UniqueName: \"kubernetes.io/projected/ee651883-6ddd-41e4-a610-9426f491a7a8-kube-api-access-92dpw\") pod \"network-check-source-869dcfcdcd-6vzg5\" (UID: \"ee651883-6ddd-41e4-a610-9426f491a7a8\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" Jul 12 14:45:23.064162 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.064002 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lnqq\" (UniqueName: \"kubernetes.io/projected/44027a77-f6d0-4165-8bf7-65cfe1babdd6-kube-api-access-8lnqq\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:23.131382 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.131346 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" Jul 12 14:45:23.144559 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.144532 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:23.152845 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.152823 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:23.156788 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.156760 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4d3fd18c-856a-42ec-a45c-2087f975bd1f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.156901 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.156802 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h667r\" (UniqueName: \"kubernetes.io/projected/4d3fd18c-856a-42ec-a45c-2087f975bd1f-kube-api-access-h667r\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.156901 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.156883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4d3fd18c-856a-42ec-a45c-2087f975bd1f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.157469 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.157429 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4d3fd18c-856a-42ec-a45c-2087f975bd1f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.159046 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.159008 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4d3fd18c-856a-42ec-a45c-2087f975bd1f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.164483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.164464 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h667r\" (UniqueName: \"kubernetes.io/projected/4d3fd18c-856a-42ec-a45c-2087f975bd1f-kube-api-access-h667r\") pod \"kube-storage-version-migrator-operator-799cb7f958-n6vd5\" (UID: \"4d3fd18c-856a-42ec-a45c-2087f975bd1f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.225534 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.225259 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" Jul 12 14:45:23.255389 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.255316 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5"] Jul 12 14:45:23.258515 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:45:23.258486 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee651883_6ddd_41e4_a610_9426f491a7a8.slice/crio-e05543a4952aa54ab2a4b32cc13cdcd2c087b0a366c68d12f63ec86992ef04b0 WatchSource:0}: Error finding container e05543a4952aa54ab2a4b32cc13cdcd2c087b0a366c68d12f63ec86992ef04b0: Status 404 returned error can't find the container with id e05543a4952aa54ab2a4b32cc13cdcd2c087b0a366c68d12f63ec86992ef04b0 Jul 12 14:45:23.275367 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.275334 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-gg4r2"] Jul 12 14:45:23.279812 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:45:23.278998 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a83e5a4_34bc_4272_b2b5_5efe836a0ef2.slice/crio-0173bbf805e2059213cca5f6aafab262f8acb91507e6ee892031b59688491353 WatchSource:0}: Error finding container 0173bbf805e2059213cca5f6aafab262f8acb91507e6ee892031b59688491353: Status 404 returned error can't find the container with id 0173bbf805e2059213cca5f6aafab262f8acb91507e6ee892031b59688491353 Jul 12 14:45:23.344363 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.344329 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gg4r2" event={"ID":"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2","Type":"ContainerStarted","Data":"0173bbf805e2059213cca5f6aafab262f8acb91507e6ee892031b59688491353"} Jul 12 14:45:23.345678 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.345653 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" event={"ID":"ee651883-6ddd-41e4-a610-9426f491a7a8","Type":"ContainerStarted","Data":"d74ed2015d7d23cac6fdb49e3504e4f52781d90ddb7827ea49e942218f76faa6"} Jul 12 14:45:23.345793 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.345691 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" event={"ID":"ee651883-6ddd-41e4-a610-9426f491a7a8","Type":"ContainerStarted","Data":"e05543a4952aa54ab2a4b32cc13cdcd2c087b0a366c68d12f63ec86992ef04b0"} Jul 12 14:45:23.353295 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.353276 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5"] Jul 12 14:45:23.356780 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:45:23.356750 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3fd18c_856a_42ec_a45c_2087f975bd1f.slice/crio-85acf6db04bea245d7392e599d3bfd8560987de1b262c120f12cab9174076115 WatchSource:0}: Error finding container 85acf6db04bea245d7392e599d3bfd8560987de1b262c120f12cab9174076115: Status 404 returned error can't find the container with id 85acf6db04bea245d7392e599d3bfd8560987de1b262c120f12cab9174076115 Jul 12 14:45:23.360080 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.359853 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-6vzg5" podStartSLOduration=1.359820514 podStartE2EDuration="1.359820514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:23.359441178 +0000 UTC m=+271.327324430" watchObservedRunningTime="2026-07-12 14:45:23.359820514 +0000 UTC m=+271.327703767" Jul 12 14:45:23.559703 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:23.559626 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:23.559839 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:23.559776 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 14:45:23.559839 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:23.559835 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls podName:44027a77-f6d0-4165-8bf7-65cfe1babdd6 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:24.559817462 +0000 UTC m=+272.527700697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls") pod "cluster-samples-operator-65db858894-7fwld" (UID: "44027a77-f6d0-4165-8bf7-65cfe1babdd6") : secret "samples-operator-tls" not found Jul 12 14:45:24.355687 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:24.355639 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" event={"ID":"4d3fd18c-856a-42ec-a45c-2087f975bd1f","Type":"ContainerStarted","Data":"85acf6db04bea245d7392e599d3bfd8560987de1b262c120f12cab9174076115"} Jul 12 14:45:24.569324 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:24.569280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:24.569517 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:24.569446 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 14:45:24.569568 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:24.569520 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls podName:44027a77-f6d0-4165-8bf7-65cfe1babdd6 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:26.569499431 +0000 UTC m=+274.537382663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls") pod "cluster-samples-operator-65db858894-7fwld" (UID: "44027a77-f6d0-4165-8bf7-65cfe1babdd6") : secret "samples-operator-tls" not found Jul 12 14:45:24.992460 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:24.992429 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:25.044620 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:25.044579 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:26.327632 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.327601 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:26.363432 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.363408 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/0.log" Jul 12 14:45:26.363566 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.363441 2573 generic.go:356] "Generic (PLEG): container finished" podID="6a83e5a4-34bc-4272-b2b5-5efe836a0ef2" containerID="d7c3b43de24c25ad6a090a25db4458492611eaa421e4fc2a5b1f959d37e44122" exitCode=255 Jul 12 14:45:26.363566 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.363511 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gg4r2" event={"ID":"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2","Type":"ContainerDied","Data":"d7c3b43de24c25ad6a090a25db4458492611eaa421e4fc2a5b1f959d37e44122"} Jul 12 14:45:26.363767 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.363752 2573 scope.go:117] "RemoveContainer" containerID="d7c3b43de24c25ad6a090a25db4458492611eaa421e4fc2a5b1f959d37e44122" Jul 12 14:45:26.365111 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.365086 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" event={"ID":"4d3fd18c-856a-42ec-a45c-2087f975bd1f","Type":"ContainerStarted","Data":"976cc703ca745334a7a3ea43acf38cc3e7abb2816c176b172876b9918d01e793"} Jul 12 14:45:26.386409 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.386382 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:26.395031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.394816 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" podStartSLOduration=2.06499038 podStartE2EDuration="4.394797808s" totalImagesPullingTime="2.329807428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:26.394473963 +0000 UTC m=+274.362357227" watchObservedRunningTime="2026-07-12 14:45:26.394797808 +0000 UTC m=+274.362681054" Jul 12 14:45:26.587263 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.587166 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:26.587408 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:26.587317 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 14:45:26.587408 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:26.587377 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls podName:44027a77-f6d0-4165-8bf7-65cfe1babdd6 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:30.587361064 +0000 UTC m=+278.555244298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls") pod "cluster-samples-operator-65db858894-7fwld" (UID: "44027a77-f6d0-4165-8bf7-65cfe1babdd6") : secret "samples-operator-tls" not found Jul 12 14:45:26.770472 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.770436 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:26.871704 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:26.871617 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:27.038737 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.038701 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:27.141852 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.141820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:27.369444 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.369417 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:45:27.369813 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.369798 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/0.log" Jul 12 14:45:27.369856 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.369830 2573 generic.go:356] "Generic (PLEG): container finished" podID="6a83e5a4-34bc-4272-b2b5-5efe836a0ef2" containerID="8f1601621fa4568278279735bc836761ce47dc11bc59b2e5237887f8ca7fedb8" exitCode=255 Jul 12 14:45:27.369917 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.369903 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gg4r2" event={"ID":"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2","Type":"ContainerDied","Data":"8f1601621fa4568278279735bc836761ce47dc11bc59b2e5237887f8ca7fedb8"} Jul 12 14:45:27.369968 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.369947 2573 scope.go:117] "RemoveContainer" containerID="d7c3b43de24c25ad6a090a25db4458492611eaa421e4fc2a5b1f959d37e44122" Jul 12 14:45:27.370147 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.370128 2573 scope.go:117] "RemoveContainer" containerID="8f1601621fa4568278279735bc836761ce47dc11bc59b2e5237887f8ca7fedb8" Jul 12 14:45:27.370315 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:27.370293 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gg4r2_openshift-console-operator(6a83e5a4-34bc-4272-b2b5-5efe836a0ef2)\"" pod="openshift-console-operator/console-operator-7866497494-gg4r2" podUID="6a83e5a4-34bc-4272-b2b5-5efe836a0ef2" Jul 12 14:45:27.438680 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.438607 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:27.535379 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.535343 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:27.896768 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.896735 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-br9f6"] Jul 12 14:45:27.900901 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.900887 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:27.903728 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.903712 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 14:45:27.904699 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.904681 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 14:45:27.904816 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.904682 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gccgz\"" Jul 12 14:45:27.904816 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.904714 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 14:45:27.904816 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.904682 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 14:45:27.931729 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.931702 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-br9f6"] Jul 12 14:45:27.978430 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.978391 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:27.999526 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.999484 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8nc2\" (UniqueName: \"kubernetes.io/projected/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-api-access-m8nc2\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:27.999708 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.999594 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/718e87e6-ce7d-4af0-ba12-b9783102d8aa-crio-socket\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:27.999708 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.999642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:27.999708 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.999664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:27.999708 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:27.999688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/718e87e6-ce7d-4af0-ba12-b9783102d8aa-data-volume\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.101040 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/718e87e6-ce7d-4af0-ba12-b9783102d8aa-crio-socket\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.101495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.101522 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/718e87e6-ce7d-4af0-ba12-b9783102d8aa-crio-socket\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.101538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.101610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/718e87e6-ce7d-4af0-ba12-b9783102d8aa-data-volume\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.101626 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.101712 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:28.601685233 +0000 UTC m=+276.569568481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : configmap "kube-rbac-proxy" not found Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.101734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8nc2\" (UniqueName: \"kubernetes.io/projected/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-api-access-m8nc2\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.102045 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/718e87e6-ce7d-4af0-ba12-b9783102d8aa-data-volume\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.102163 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 14:45:28.104929 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.102758 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:28.602734414 +0000 UTC m=+276.570617662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : secret "insights-runtime-extractor-tls" not found Jul 12 14:45:28.110652 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.110625 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8nc2\" (UniqueName: \"kubernetes.io/projected/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-api-access-m8nc2\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.374064 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.373970 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:45:28.374485 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.374299 2573 scope.go:117] "RemoveContainer" containerID="8f1601621fa4568278279735bc836761ce47dc11bc59b2e5237887f8ca7fedb8" Jul 12 14:45:28.374485 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.374459 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gg4r2_openshift-console-operator(6a83e5a4-34bc-4272-b2b5-5efe836a0ef2)\"" pod="openshift-console-operator/console-operator-7866497494-gg4r2" podUID="6a83e5a4-34bc-4272-b2b5-5efe836a0ef2" Jul 12 14:45:28.475206 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.475171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:28.530446 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.530411 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:28.607097 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.607066 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.607097 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.607099 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:28.607278 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.607179 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 14:45:28.607278 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.607217 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 14:45:28.607278 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.607249 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:29.607233084 +0000 UTC m=+277.575116315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : configmap "kube-rbac-proxy" not found Jul 12 14:45:28.607278 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:28.607262 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:29.607256663 +0000 UTC m=+277.575139894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : secret "insights-runtime-extractor-tls" not found Jul 12 14:45:28.634659 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.634623 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-scs22"] Jul 12 14:45:28.638354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.638337 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.640251 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.640224 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 14:45:28.640377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.640224 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 14:45:28.640377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.640235 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 14:45:28.640377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.640339 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 14:45:28.640489 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.640463 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-kthm8\"" Jul 12 14:45:28.646303 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.646281 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-scs22"] Jul 12 14:45:28.671406 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.671373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:28.707823 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.707796 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggd2q\" (UniqueName: \"kubernetes.io/projected/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-kube-api-access-ggd2q\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.707962 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.707830 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-signing-cabundle\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.707962 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.707869 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-signing-key\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.726204 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.726177 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:28.808850 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.808821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-signing-key\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.808969 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.808927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggd2q\" (UniqueName: \"kubernetes.io/projected/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-kube-api-access-ggd2q\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.808969 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.808951 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-signing-cabundle\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.809554 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.809529 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-signing-cabundle\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.811229 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.811207 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-signing-key\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.816732 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.816710 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggd2q\" (UniqueName: \"kubernetes.io/projected/f74eaa95-c17c-44cf-b8a5-24c03d01d68d-kube-api-access-ggd2q\") pod \"service-ca-79f68b9f4f-scs22\" (UID: \"f74eaa95-c17c-44cf-b8a5-24c03d01d68d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:28.947328 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:28.947224 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" Jul 12 14:45:29.062285 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:29.062253 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-scs22"] Jul 12 14:45:29.065338 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:45:29.065313 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf74eaa95_c17c_44cf_b8a5_24c03d01d68d.slice/crio-e1b55d5feb48d01936fc6bcb0c6e1ce711f47af8ea3da93ec3695d6194354f2a WatchSource:0}: Error finding container e1b55d5feb48d01936fc6bcb0c6e1ce711f47af8ea3da93ec3695d6194354f2a: Status 404 returned error can't find the container with id e1b55d5feb48d01936fc6bcb0c6e1ce711f47af8ea3da93ec3695d6194354f2a Jul 12 14:45:29.378799 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:29.378770 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" event={"ID":"f74eaa95-c17c-44cf-b8a5-24c03d01d68d","Type":"ContainerStarted","Data":"e1b55d5feb48d01936fc6bcb0c6e1ce711f47af8ea3da93ec3695d6194354f2a"} Jul 12 14:45:29.579827 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:29.579792 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:29.616037 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:29.615989 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:29.616226 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:29.616042 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:29.616226 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:29.616112 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 14:45:29.616226 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:29.616169 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 14:45:29.616226 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:29.616178 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:31.616159964 +0000 UTC m=+279.584043211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : configmap "kube-rbac-proxy" not found Jul 12 14:45:29.616226 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:29.616205 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:31.616193717 +0000 UTC m=+279.584076950 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : secret "insights-runtime-extractor-tls" not found Jul 12 14:45:30.088919 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:30.088887 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ppcrh_7aa6132f-7dbf-439b-8683-2d1aca184a98/dns-node-resolver/0.log" Jul 12 14:45:30.624406 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:30.624363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:30.624843 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:30.624530 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 14:45:30.624843 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:30.624601 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls podName:44027a77-f6d0-4165-8bf7-65cfe1babdd6 nodeName:}" failed. No retries permitted until 2026-07-12 14:45:38.624582633 +0000 UTC m=+286.592465924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls") pod "cluster-samples-operator-65db858894-7fwld" (UID: "44027a77-f6d0-4165-8bf7-65cfe1babdd6") : secret "samples-operator-tls" not found Jul 12 14:45:30.888318 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:30.888245 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6mbxr_3043ff23-e0c3-4c6a-80a0-80566bc79d76/node-ca/0.log" Jul 12 14:45:31.386400 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:31.386367 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" event={"ID":"f74eaa95-c17c-44cf-b8a5-24c03d01d68d","Type":"ContainerStarted","Data":"8d992a32e0678277787fb3661f06369ad85e55c3d131e2436eaa7a810535bf8d"} Jul 12 14:45:31.400472 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:31.400420 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-scs22" podStartSLOduration=1.574733927 podStartE2EDuration="3.400402657s" totalImagesPullingTime="1.82566873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:31.400043777 +0000 UTC m=+279.367927029" watchObservedRunningTime="2026-07-12 14:45:31.400402657 +0000 UTC m=+279.368285914" Jul 12 14:45:31.489120 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:31.489088 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ckf4h_b4aa18e4-0242-415b-90bc-af206b09567f/serve-healthcheck-canary/0.log" Jul 12 14:45:31.632924 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:31.632881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:31.633322 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:31.632929 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:31.633322 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:31.632994 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 14:45:31.633322 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:31.633069 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:35.633054462 +0000 UTC m=+283.600937694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : configmap "kube-rbac-proxy" not found Jul 12 14:45:31.633322 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:31.633097 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 14:45:31.633322 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:31.633144 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:35.633130139 +0000 UTC m=+283.601013378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : secret "insights-runtime-extractor-tls" not found Jul 12 14:45:32.274673 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:32.274635 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:32.376788 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:32.376710 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:32.477541 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:32.477511 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:32.579055 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:32.578943 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:32.782702 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:32.782671 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:32.882757 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:32.882720 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:32.987230 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:32.987189 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:33.145152 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:33.145070 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:33.145152 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:33.145123 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:33.145539 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:33.145523 2573 scope.go:117] "RemoveContainer" containerID="8f1601621fa4568278279735bc836761ce47dc11bc59b2e5237887f8ca7fedb8" Jul 12 14:45:33.145725 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:33.145694 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gg4r2_openshift-console-operator(6a83e5a4-34bc-4272-b2b5-5efe836a0ef2)\"" pod="openshift-console-operator/console-operator-7866497494-gg4r2" podUID="6a83e5a4-34bc-4272-b2b5-5efe836a0ef2" Jul 12 14:45:33.980637 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:33.980602 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:34.071235 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:34.071188 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" podUID="10e3433a-1692-4ea9-8e91-be8d0739b2d6" Jul 12 14:45:34.071235 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:34.071218 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4rjb5" podUID="d1e45d8b-95d2-4193-96df-9749957d03a7" Jul 12 14:45:34.071235 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:34.071218 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" Jul 12 14:45:34.197161 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:34.197133 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:34.396172 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:34.396139 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:45:34.396377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:34.396139 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:45:34.396377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:34.396153 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rjb5" Jul 12 14:45:34.584432 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:34.584397 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:34.788942 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:34.788866 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:35.374278 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:35.374248 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:35.667463 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:35.667361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:35.667463 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:35.667408 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:35.667658 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:35.667486 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 14:45:35.667658 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:35.667507 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 14:45:35.667658 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:35.667567 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:43.667550814 +0000 UTC m=+291.635434049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : configmap "kube-rbac-proxy" not found Jul 12 14:45:35.667658 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:35.667583 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls podName:718e87e6-ce7d-4af0-ba12-b9783102d8aa nodeName:}" failed. No retries permitted until 2026-07-12 14:45:43.667577272 +0000 UTC m=+291.635460503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br9f6" (UID: "718e87e6-ce7d-4af0-ba12-b9783102d8aa") : secret "insights-runtime-extractor-tls" not found Jul 12 14:45:35.980133 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:35.980041 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:36.031149 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:36.031118 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:36.181229 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:36.181195 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:36.235340 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:36.235264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:36.586060 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:36.585967 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:36.782220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:36.782188 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:37.179275 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.179229 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:37.481123 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.481041 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:45:37.481291 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.481142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:45:37.481291 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.481166 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:45:37.481291 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:37.481270 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 14:45:37.481452 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:37.481333 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert podName:10e3433a-1692-4ea9-8e91-be8d0739b2d6 nodeName:}" failed. No retries permitted until 2026-07-12 14:47:39.481316162 +0000 UTC m=+407.449199397 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-pstf2" (UID: "10e3433a-1692-4ea9-8e91-be8d0739b2d6") : secret "networking-console-plugin-cert" not found Jul 12 14:45:37.483434 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.483407 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1e45d8b-95d2-4193-96df-9749957d03a7-metrics-tls\") pod \"dns-default-4rjb5\" (UID: \"d1e45d8b-95d2-4193-96df-9749957d03a7\") " pod="openshift-dns/dns-default-4rjb5" Jul 12 14:45:37.483545 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.483518 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"image-registry-765998dfdb-wtmkw\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:45:37.701769 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.701737 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ff579\"" Jul 12 14:45:37.702299 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.702285 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zmjmj\"" Jul 12 14:45:37.707135 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.707112 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rjb5" Jul 12 14:45:37.707233 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.707198 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:45:37.785272 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.785216 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:37.848375 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.848307 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-765998dfdb-wtmkw"] Jul 12 14:45:37.850997 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:45:37.850971 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881b7e06_c94e_4d19_8f3d_da8a15f8efb1.slice/crio-3142df3f29de4cbbe8cca5e29ee7092a7073201e8f357dcfcebeadce8553250e WatchSource:0}: Error finding container 3142df3f29de4cbbe8cca5e29ee7092a7073201e8f357dcfcebeadce8553250e: Status 404 returned error can't find the container with id 3142df3f29de4cbbe8cca5e29ee7092a7073201e8f357dcfcebeadce8553250e Jul 12 14:45:37.859578 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.859552 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rjb5"] Jul 12 14:45:37.863365 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:45:37.863339 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e45d8b_95d2_4193_96df_9749957d03a7.slice/crio-e4ed98315e270c82bb4ff0e33af5d4da54ee60f94932bb2b313b136e7185760e WatchSource:0}: Error finding container e4ed98315e270c82bb4ff0e33af5d4da54ee60f94932bb2b313b136e7185760e: Status 404 returned error can't find the container with id e4ed98315e270c82bb4ff0e33af5d4da54ee60f94932bb2b313b136e7185760e Jul 12 14:45:37.976336 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:37.976305 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:38.408589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.408549 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" event={"ID":"881b7e06-c94e-4d19-8f3d-da8a15f8efb1","Type":"ContainerStarted","Data":"c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916"} Jul 12 14:45:38.408796 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.408648 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" event={"ID":"881b7e06-c94e-4d19-8f3d-da8a15f8efb1","Type":"ContainerStarted","Data":"3142df3f29de4cbbe8cca5e29ee7092a7073201e8f357dcfcebeadce8553250e"} Jul 12 14:45:38.408796 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.408677 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:45:38.409563 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.409541 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rjb5" event={"ID":"d1e45d8b-95d2-4193-96df-9749957d03a7","Type":"ContainerStarted","Data":"e4ed98315e270c82bb4ff0e33af5d4da54ee60f94932bb2b313b136e7185760e"} Jul 12 14:45:38.426375 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.426322 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" podStartSLOduration=285.426306311 podStartE2EDuration="4m45.426306311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:38.425726354 +0000 UTC m=+286.393609606" watchObservedRunningTime="2026-07-12 14:45:38.426306311 +0000 UTC m=+286.394189557" Jul 12 14:45:38.576937 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.576899 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:38.693041 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.692939 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:38.695728 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.695699 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/44027a77-f6d0-4165-8bf7-65cfe1babdd6-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-7fwld\" (UID: \"44027a77-f6d0-4165-8bf7-65cfe1babdd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:38.739023 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.738971 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" Jul 12 14:45:38.765284 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.765251 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:38.878217 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:38.878185 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld"] Jul 12 14:45:39.181377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:39.181338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:39.377527 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:39.377494 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:39.413787 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:39.413748 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" event={"ID":"44027a77-f6d0-4165-8bf7-65cfe1babdd6","Type":"ContainerStarted","Data":"5847bd5254f261b97ef9fcfad4d3e28d39ab2c5bd7637325c0e904ac346b03d1"} Jul 12 14:45:39.577616 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:39.577497 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:39.979467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:39.979429 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:40.079292 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:40.079257 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:40.418795 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:40.418763 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rjb5" event={"ID":"d1e45d8b-95d2-4193-96df-9749957d03a7","Type":"ContainerStarted","Data":"2439eceae3b3578c29a81eb97498d4040085fb22e5bfcf58e4335520566680c8"} Jul 12 14:45:40.418982 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:40.418814 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rjb5" event={"ID":"d1e45d8b-95d2-4193-96df-9749957d03a7","Type":"ContainerStarted","Data":"d3505dc0750031f79f111d404b9beb785006de6cc3dcc448f651c5f7ed9f7232"} Jul 12 14:45:40.418982 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:40.418857 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4rjb5" Jul 12 14:45:40.435888 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:40.435829 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4rjb5" podStartSLOduration=251.709432282 podStartE2EDuration="4m13.435809983s" totalImagesPullingTime="1.726377701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:40.43511968 +0000 UTC m=+288.403002929" watchObservedRunningTime="2026-07-12 14:45:40.435809983 +0000 UTC m=+288.403693236" Jul 12 14:45:40.574690 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:40.574659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:40.781284 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:40.781188 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:41.175886 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.175851 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:41.423083 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.423044 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" event={"ID":"44027a77-f6d0-4165-8bf7-65cfe1babdd6","Type":"ContainerStarted","Data":"dab9e12f931b3b7c2c1ad1fb694abf7d77ba3051455cfaf38a6a0862b04c6030"} Jul 12 14:45:41.423278 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.423112 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" event={"ID":"44027a77-f6d0-4165-8bf7-65cfe1babdd6","Type":"ContainerStarted","Data":"eb10c23751921a34e2cd65fd7e1942186790556baeccf38edd291f0a6b313236"} Jul 12 14:45:41.438156 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.438059 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-7fwld" podStartSLOduration=17.495769453 podStartE2EDuration="19.438040011s" totalImagesPullingTime="1.942270558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:41.437628501 +0000 UTC m=+289.405511755" watchObservedRunningTime="2026-07-12 14:45:41.438040011 +0000 UTC m=+289.405923267" Jul 12 14:45:41.784468 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.784382 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:41.835056 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.835029 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:41.894427 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.894392 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:41.979649 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:41.979614 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:42.377615 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:42.377585 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:42.580354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:42.580310 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:42.784264 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:42.784181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:42.973268 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:42.973232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:43.731737 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:43.731694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:43.731737 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:43.731735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:43.732445 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:43.732416 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/718e87e6-ce7d-4af0-ba12-b9783102d8aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:43.733904 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:43.733877 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/718e87e6-ce7d-4af0-ba12-b9783102d8aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br9f6\" (UID: \"718e87e6-ce7d-4af0-ba12-b9783102d8aa\") " pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:43.809605 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:43.809573 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-br9f6" Jul 12 14:45:43.933061 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:43.933030 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-br9f6"] Jul 12 14:45:43.936208 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:45:43.936181 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod718e87e6_ce7d_4af0_ba12_b9783102d8aa.slice/crio-d77c1e60c9c4e1db5ced49100875ff595f6484c6752c7038c713dfd53061b867 WatchSource:0}: Error finding container d77c1e60c9c4e1db5ced49100875ff595f6484c6752c7038c713dfd53061b867: Status 404 returned error can't find the container with id d77c1e60c9c4e1db5ced49100875ff595f6484c6752c7038c713dfd53061b867 Jul 12 14:45:44.435960 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:44.435929 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br9f6" event={"ID":"718e87e6-ce7d-4af0-ba12-b9783102d8aa","Type":"ContainerStarted","Data":"d77c1e60c9c4e1db5ced49100875ff595f6484c6752c7038c713dfd53061b867"} Jul 12 14:45:46.572079 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:45:46.572027 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fjw4t" podUID="9ab38523-f4fb-4fb3-9111-4dd8070e71de" Jul 12 14:45:47.570512 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:47.570485 2573 scope.go:117] "RemoveContainer" containerID="8f1601621fa4568278279735bc836761ce47dc11bc59b2e5237887f8ca7fedb8" Jul 12 14:45:48.453091 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:48.452254 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:45:48.453091 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:48.452362 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gg4r2" event={"ID":"6a83e5a4-34bc-4272-b2b5-5efe836a0ef2","Type":"ContainerStarted","Data":"d6fd592375d626b8dbe54b750f53867fbbf58804d867168825cd97d61d1a4abb"} Jul 12 14:45:48.453549 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:48.453182 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:48.456245 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:48.456194 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br9f6" event={"ID":"718e87e6-ce7d-4af0-ba12-b9783102d8aa","Type":"ContainerStarted","Data":"9194ec81904006615706c3162dce8b94cd8ba1607c47920550e95cedaa6c5576"} Jul 12 14:45:48.456245 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:48.456244 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br9f6" event={"ID":"718e87e6-ce7d-4af0-ba12-b9783102d8aa","Type":"ContainerStarted","Data":"7de2ed86cc2964ae6c450d378251b4ac36f453ae48112d6657c97c238bdccb94"} Jul 12 14:45:48.469817 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:48.469738 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-gg4r2" podStartSLOduration=24.064852057 podStartE2EDuration="26.46972328s" totalImagesPullingTime="2.404871223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:48.468196585 +0000 UTC m=+296.436079838" watchObservedRunningTime="2026-07-12 14:45:48.46972328 +0000 UTC m=+296.437606532" Jul 12 14:45:49.041961 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:49.041930 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-gg4r2" Jul 12 14:45:49.220375 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:49.220326 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:49.321415 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:49.321335 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:49.378319 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:49.378284 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:49.515436 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:49.515398 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:49.694898 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:49.694863 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:49.891118 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:49.891086 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:50.316873 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:50.316814 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:50.374569 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:50.374536 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:50.426165 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:50.426133 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4rjb5" Jul 12 14:45:50.608675 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:50.608595 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:50.704148 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:50.704110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:50.829182 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:50.829151 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:51.098241 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:51.098211 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:51.303321 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:51.303243 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:51.469510 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:51.469467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br9f6" event={"ID":"718e87e6-ce7d-4af0-ba12-b9783102d8aa","Type":"ContainerStarted","Data":"161bf871a5b3ea92ca334d23713d809e048897741bba606cc9d17d01aea0e3c7"} Jul 12 14:45:51.480389 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:51.480362 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:51.485598 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:51.485558 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-br9f6" podStartSLOduration=17.670453768 podStartE2EDuration="24.485540133s" totalImagesPullingTime="6.815086365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 14:45:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:45:51.48471448 +0000 UTC m=+299.452597733" watchObservedRunningTime="2026-07-12 14:45:51.485540133 +0000 UTC m=+299.453423386" Jul 12 14:45:52.458745 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:52.458718 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:45:52.459278 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:52.459125 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:45:52.465459 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:52.465437 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:45:52.465963 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:52.465941 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:45:52.472908 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:52.471136 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 14:45:52.722637 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:52.722555 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:52.777298 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:52.777267 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:57.074385 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:57.074345 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:57.712432 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:57.712399 2573 patch_prober.go:28] interesting pod/image-registry-765998dfdb-wtmkw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 12 14:45:57.712610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:57.712451 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 14:45:57.967561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:57.967469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:59.418830 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:59.418801 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:45:59.642434 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:59.642401 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:59.739798 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:59.739714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:45:59.928984 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:45:59.928951 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77d65bff77-nwkqb"] Jul 12 14:46:00.109300 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.109208 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77d65bff77-nwkqb"] Jul 12 14:46:00.109472 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.109336 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.111886 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.111858 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 14:46:00.112272 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.112251 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 14:46:00.112425 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.112282 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 14:46:00.112425 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.112416 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 14:46:00.112598 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.112305 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 14:46:00.112598 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.112324 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 14:46:00.112598 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.112305 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 14:46:00.112598 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.112356 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z2l6d\"" Jul 12 14:46:00.146075 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.146035 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-oauth-serving-cert\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.146075 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.146077 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-serving-cert\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.146277 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.146170 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-oauth-config\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.146277 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.146198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52kvj\" (UniqueName: \"kubernetes.io/projected/b648a5d2-d158-4008-9020-f635ff751627-kube-api-access-52kvj\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.146277 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.146231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-console-config\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.146277 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.146274 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-service-ca\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.238610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.238569 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:00.246814 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.246781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-service-ca\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.246814 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.246816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-oauth-serving-cert\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.247098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.246838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-serving-cert\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.247098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.246974 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-oauth-config\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.247098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.247031 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52kvj\" (UniqueName: \"kubernetes.io/projected/b648a5d2-d158-4008-9020-f635ff751627-kube-api-access-52kvj\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.247098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.247081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-console-config\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.247701 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.247679 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-console-config\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.247819 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.247707 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-service-ca\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.247819 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.247707 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-oauth-serving-cert\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.249301 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.249281 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-serving-cert\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.249534 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.249514 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-oauth-config\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.254669 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.254648 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52kvj\" (UniqueName: \"kubernetes.io/projected/b648a5d2-d158-4008-9020-f635ff751627-kube-api-access-52kvj\") pod \"console-77d65bff77-nwkqb\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.274767 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.274742 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cxntn"] Jul 12 14:46:00.292663 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.292628 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cxntn"] Jul 12 14:46:00.292806 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.292792 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.294623 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.294601 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 12 14:46:00.294777 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.294756 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 12 14:46:00.294878 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.294842 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 14:46:00.294999 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.294982 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 14:46:00.295114 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.295076 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-sx9lr\"" Jul 12 14:46:00.295182 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.295121 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 14:46:00.309998 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.309973 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:00.348301 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.348269 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.348408 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.348343 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swh4x\" (UniqueName: \"kubernetes.io/projected/e381d1ab-46c6-4822-9f57-e80d038ed4bf-kube-api-access-swh4x\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.348408 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.348388 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.348484 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.348446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e381d1ab-46c6-4822-9f57-e80d038ed4bf-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.418444 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.418413 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:00.449234 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.449198 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swh4x\" (UniqueName: \"kubernetes.io/projected/e381d1ab-46c6-4822-9f57-e80d038ed4bf-kube-api-access-swh4x\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.449572 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.449253 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.449572 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.449304 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e381d1ab-46c6-4822-9f57-e80d038ed4bf-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.449572 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.449368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.449572 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:46:00.449416 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 12 14:46:00.449572 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:46:00.449482 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-tls podName:e381d1ab-46c6-4822-9f57-e80d038ed4bf nodeName:}" failed. No retries permitted until 2026-07-12 14:46:00.949464004 +0000 UTC m=+308.917347242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-tls") pod "prometheus-operator-64785b6675-cxntn" (UID: "e381d1ab-46c6-4822-9f57-e80d038ed4bf") : secret "prometheus-operator-tls" not found Jul 12 14:46:00.450162 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.450142 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e381d1ab-46c6-4822-9f57-e80d038ed4bf-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.451663 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.451642 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.457176 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.457155 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swh4x\" (UniqueName: \"kubernetes.io/projected/e381d1ab-46c6-4822-9f57-e80d038ed4bf-kube-api-access-swh4x\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.530730 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.530699 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:00.560344 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.560313 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77d65bff77-nwkqb"] Jul 12 14:46:00.563007 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:46:00.562969 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb648a5d2_d158_4008_9020_f635ff751627.slice/crio-d4509a3c9e29a06963d8c6f549619269b7b8817325dbcade871fa30775f6348e WatchSource:0}: Error finding container d4509a3c9e29a06963d8c6f549619269b7b8817325dbcade871fa30775f6348e: Status 404 returned error can't find the container with id d4509a3c9e29a06963d8c6f549619269b7b8817325dbcade871fa30775f6348e Jul 12 14:46:00.565046 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.565029 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 14:46:00.570757 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.570735 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:46:00.953031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.952976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:00.955290 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:00.955271 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e381d1ab-46c6-4822-9f57-e80d038ed4bf-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cxntn\" (UID: \"e381d1ab-46c6-4822-9f57-e80d038ed4bf\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:01.202291 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:01.202250 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" Jul 12 14:46:01.382892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:01.382762 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cxntn"] Jul 12 14:46:01.386158 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:46:01.386129 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode381d1ab_46c6_4822_9f57_e80d038ed4bf.slice/crio-0cacc4979bd9407255c874014ba0e3dd6d1676c03361af377ef9d82df868f183 WatchSource:0}: Error finding container 0cacc4979bd9407255c874014ba0e3dd6d1676c03361af377ef9d82df868f183: Status 404 returned error can't find the container with id 0cacc4979bd9407255c874014ba0e3dd6d1676c03361af377ef9d82df868f183 Jul 12 14:46:01.505496 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:01.505411 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" event={"ID":"e381d1ab-46c6-4822-9f57-e80d038ed4bf","Type":"ContainerStarted","Data":"0cacc4979bd9407255c874014ba0e3dd6d1676c03361af377ef9d82df868f183"} Jul 12 14:46:01.506720 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:01.506693 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d65bff77-nwkqb" event={"ID":"b648a5d2-d158-4008-9020-f635ff751627","Type":"ContainerStarted","Data":"d4509a3c9e29a06963d8c6f549619269b7b8817325dbcade871fa30775f6348e"} Jul 12 14:46:04.006288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.006232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:04.107003 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.106969 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:04.171839 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.171807 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:04.262579 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.262500 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:04.364208 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.364176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:04.519149 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.519084 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" event={"ID":"e381d1ab-46c6-4822-9f57-e80d038ed4bf","Type":"ContainerStarted","Data":"4b76971d3644a0728e6e3d0a455b32fdc43eebef03256ba769aa93c04c9a8c5e"} Jul 12 14:46:04.520736 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.520713 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d65bff77-nwkqb" event={"ID":"b648a5d2-d158-4008-9020-f635ff751627","Type":"ContainerStarted","Data":"d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9"} Jul 12 14:46:04.536242 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.536188 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77d65bff77-nwkqb" podStartSLOduration=1.719972227 podStartE2EDuration="5.536167914s" totalImagesPullingTime="3.816195687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:45:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:46:04.53570645 +0000 UTC m=+312.503589702" watchObservedRunningTime="2026-07-12 14:46:04.536167914 +0000 UTC m=+312.504051168" Jul 12 14:46:04.725288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:04.725257 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:05.525438 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:05.525404 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" event={"ID":"e381d1ab-46c6-4822-9f57-e80d038ed4bf","Type":"ContainerStarted","Data":"a1413f3c53294b13b8a9fb72b523794297d5b191e5c94152fed7d743722f28f5"} Jul 12 14:46:05.541398 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:05.541352 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-cxntn" podStartSLOduration=2.546652175 podStartE2EDuration="5.541337353s" totalImagesPullingTime="2.994685178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:46:05.540098478 +0000 UTC m=+313.507981731" watchObservedRunningTime="2026-07-12 14:46:05.541337353 +0000 UTC m=+313.509220605" Jul 12 14:46:06.127249 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:06.127215 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:06.183867 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:06.183840 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:07.364593 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:07.364558 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:07.799618 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:07.799586 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:10.121218 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.121186 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:10.222236 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.222201 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:10.418891 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.418847 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:10.419103 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.418909 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:10.423589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.423564 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:10.526712 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.526682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:10.549117 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.549086 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:10.627406 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:10.627379 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:11.707125 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.707090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:11.746838 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.746805 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lpggh"] Jul 12 14:46:11.756331 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.756295 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.760769 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.760739 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 14:46:11.760962 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.760923 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 14:46:11.762804 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.762782 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mbqzh\"" Jul 12 14:46:11.764262 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.764197 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 14:46:11.840940 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.840905 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-root\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.840940 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.840941 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e433612d-2ea8-4494-bb5e-92e37affccbb-metrics-client-ca\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.841212 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.840965 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-textfile\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.841212 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.841105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd9gp\" (UniqueName: \"kubernetes.io/projected/e433612d-2ea8-4494-bb5e-92e37affccbb-kube-api-access-xd9gp\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.841212 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.841145 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-accelerators-collector-config\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.841357 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.841223 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-tls\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.841357 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.841274 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-sys\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.841357 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.841296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.841357 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.841330 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-wtmp\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.860107 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.860066 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:11.942215 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942178 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-tls\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942407 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-sys\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942407 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942407 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-wtmp\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942407 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942319 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-sys\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942407 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:46:11.942360 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 12 14:46:11.942407 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942374 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-root\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e433612d-2ea8-4494-bb5e-92e37affccbb-metrics-client-ca\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942695 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:46:11.942433 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-tls podName:e433612d-2ea8-4494-bb5e-92e37affccbb nodeName:}" failed. No retries permitted until 2026-07-12 14:46:12.442412737 +0000 UTC m=+320.410295975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-tls") pod "node-exporter-lpggh" (UID: "e433612d-2ea8-4494-bb5e-92e37affccbb") : secret "node-exporter-tls" not found Jul 12 14:46:11.942695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942461 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-wtmp\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942463 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-textfile\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd9gp\" (UniqueName: \"kubernetes.io/projected/e433612d-2ea8-4494-bb5e-92e37affccbb-kube-api-access-xd9gp\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-accelerators-collector-config\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.942981 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.942832 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e433612d-2ea8-4494-bb5e-92e37affccbb-root\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.943067 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.943040 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e433612d-2ea8-4494-bb5e-92e37affccbb-metrics-client-ca\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.943187 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.943150 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-accelerators-collector-config\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.943187 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.943173 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-textfile\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.944797 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.944773 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.949982 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.949955 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd9gp\" (UniqueName: \"kubernetes.io/projected/e433612d-2ea8-4494-bb5e-92e37affccbb-kube-api-access-xd9gp\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:11.960007 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:11.959935 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.060746 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.060665 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.160978 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.160944 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.260461 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.260386 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.360347 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.360315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.448039 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.447977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-tls\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:12.450340 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.450317 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e433612d-2ea8-4494-bb5e-92e37affccbb-node-exporter-tls\") pod \"node-exporter-lpggh\" (UID: \"e433612d-2ea8-4494-bb5e-92e37affccbb\") " pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:12.604845 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.604758 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-765998dfdb-wtmkw"] Jul 12 14:46:12.668611 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.668575 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lpggh" Jul 12 14:46:12.678220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.678191 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.679004 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:46:12.678982 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode433612d_2ea8_4494_bb5e_92e37affccbb.slice/crio-d80d623aecc09699fa6cca2da63fd7cd884ed215f6e804746e8b89975e9b9f8d WatchSource:0}: Error finding container d80d623aecc09699fa6cca2da63fd7cd884ed215f6e804746e8b89975e9b9f8d: Status 404 returned error can't find the container with id d80d623aecc09699fa6cca2da63fd7cd884ed215f6e804746e8b89975e9b9f8d Jul 12 14:46:12.782970 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.782935 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.856005 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.855912 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.910811 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.910764 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:12.997723 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:12.997689 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:13.077631 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:13.077600 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:13.401027 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:13.400978 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:13.455279 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:13.455239 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:13.556355 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:13.556327 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lpggh" event={"ID":"e433612d-2ea8-4494-bb5e-92e37affccbb","Type":"ContainerStarted","Data":"d80d623aecc09699fa6cca2da63fd7cd884ed215f6e804746e8b89975e9b9f8d"} Jul 12 14:46:14.561081 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:14.561038 2573 generic.go:356] "Generic (PLEG): container finished" podID="e433612d-2ea8-4494-bb5e-92e37affccbb" containerID="a9a2aebe9dc905405feb76410b2946cc94b688ecb012f48410e8c01563895478" exitCode=0 Jul 12 14:46:14.561486 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:14.561101 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lpggh" event={"ID":"e433612d-2ea8-4494-bb5e-92e37affccbb","Type":"ContainerDied","Data":"a9a2aebe9dc905405feb76410b2946cc94b688ecb012f48410e8c01563895478"} Jul 12 14:46:14.702517 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:14.702477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:14.802386 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:14.802344 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:14.971749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:14.971698 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:15.566764 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:15.566729 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lpggh" event={"ID":"e433612d-2ea8-4494-bb5e-92e37affccbb","Type":"ContainerStarted","Data":"29a5c88cb413490a64833a032c53acf51fb779f78bb77b5742e260901ca31a56"} Jul 12 14:46:15.567287 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:15.566806 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lpggh" event={"ID":"e433612d-2ea8-4494-bb5e-92e37affccbb","Type":"ContainerStarted","Data":"725b59eaccf22910dc0ba10765f4525aa01e6df302da0d73651d4ca9c0335e37"} Jul 12 14:46:15.588032 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:15.587960 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lpggh" podStartSLOduration=3.680521891 podStartE2EDuration="4.587942767s" totalImagesPullingTime="843.942833ms" totalInitContainerRuntime="63.478043ms" isStatefulPod=true podCreationTimestamp="2026-07-12 14:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:46:15.586180473 +0000 UTC m=+323.554063726" watchObservedRunningTime="2026-07-12 14:46:15.587942767 +0000 UTC m=+323.555826087" Jul 12 14:46:16.530461 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.530418 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:16.611799 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.611761 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:16.686822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.686791 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:16.995128 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.995097 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz"] Jul 12 14:46:16.997168 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.997148 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:16.999181 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.999161 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 12 14:46:16.999181 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.999166 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 12 14:46:16.999365 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.999267 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 12 14:46:16.999433 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.999368 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 12 14:46:16.999486 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.999451 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xchrv\"" Jul 12 14:46:16.999830 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:16.999814 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 12 14:46:17.004706 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.004684 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 12 14:46:17.013507 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.013486 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz"] Jul 12 14:46:17.088149 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.088149 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088151 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-telemeter-client-tls\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.088352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088216 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-secret-telemeter-client\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.088352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-federate-client-tls\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.088352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088299 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-serving-certs-ca-bundle\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.088352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088345 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-metrics-client-ca\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.088473 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088369 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.088473 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.088403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r8tm\" (UniqueName: \"kubernetes.io/projected/25c71d36-7402-4dbd-b893-f4524c9d7361-kube-api-access-2r8tm\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.189663 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189618 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-secret-telemeter-client\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.189844 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189671 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-federate-client-tls\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.189844 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189707 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-serving-certs-ca-bundle\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.189844 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-metrics-client-ca\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.189844 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189752 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.189844 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8tm\" (UniqueName: \"kubernetes.io/projected/25c71d36-7402-4dbd-b893-f4524c9d7361-kube-api-access-2r8tm\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.190134 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189928 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.190134 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.189984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-telemeter-client-tls\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.190600 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.190574 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-metrics-client-ca\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.190725 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.190609 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-serving-certs-ca-bundle\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.191059 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.191039 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25c71d36-7402-4dbd-b893-f4524c9d7361-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.192422 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.192399 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-telemeter-client-tls\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.192540 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.192513 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-federate-client-tls\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.192670 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.192654 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-secret-telemeter-client\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.192712 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.192671 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25c71d36-7402-4dbd-b893-f4524c9d7361-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.205839 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.205819 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r8tm\" (UniqueName: \"kubernetes.io/projected/25c71d36-7402-4dbd-b893-f4524c9d7361-kube-api-access-2r8tm\") pod \"telemeter-client-6b96bf6f66-2kxjz\" (UID: \"25c71d36-7402-4dbd-b893-f4524c9d7361\") " pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.307217 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.307128 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" Jul 12 14:46:17.429159 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.429126 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz"] Jul 12 14:46:17.432459 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:46:17.432433 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c71d36_7402_4dbd_b893_f4524c9d7361.slice/crio-bb9de4f7a8803ef4c3086701121591ea1deb01061b7df94faf197a935010bffd WatchSource:0}: Error finding container bb9de4f7a8803ef4c3086701121591ea1deb01061b7df94faf197a935010bffd: Status 404 returned error can't find the container with id bb9de4f7a8803ef4c3086701121591ea1deb01061b7df94faf197a935010bffd Jul 12 14:46:17.574798 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.574708 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" event={"ID":"25c71d36-7402-4dbd-b893-f4524c9d7361","Type":"ContainerStarted","Data":"bb9de4f7a8803ef4c3086701121591ea1deb01061b7df94faf197a935010bffd"} Jul 12 14:46:17.928609 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:17.928555 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:18.140459 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:18.140423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:19.585518 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:19.585474 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" event={"ID":"25c71d36-7402-4dbd-b893-f4524c9d7361","Type":"ContainerStarted","Data":"ff5c62e347c885e99751870db763d911768386fcc47288317f8d1bc7acdb63ee"} Jul 12 14:46:20.590596 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:20.590558 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" event={"ID":"25c71d36-7402-4dbd-b893-f4524c9d7361","Type":"ContainerStarted","Data":"2fa7f48918893f80e288c827d4d709368845d50fbb8e7175ecd0ef3ed3f6b10b"} Jul 12 14:46:20.591097 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:20.590621 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" event={"ID":"25c71d36-7402-4dbd-b893-f4524c9d7361","Type":"ContainerStarted","Data":"b03402a9e5d74653759bb1b23c9aa61e29110172232a7b309745a2e62d7a5726"} Jul 12 14:46:20.609950 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:20.609890 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6b96bf6f66-2kxjz" podStartSLOduration=2.273455152 podStartE2EDuration="4.609875589s" totalImagesPullingTime="2.336420437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:46:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:46:20.608834965 +0000 UTC m=+328.576718218" watchObservedRunningTime="2026-07-12 14:46:20.609875589 +0000 UTC m=+328.577758842" Jul 12 14:46:21.446530 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.446498 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dff484f7c-nbn8w"] Jul 12 14:46:21.448623 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.448600 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.455176 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.455152 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 12 14:46:21.460407 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.460384 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dff484f7c-nbn8w"] Jul 12 14:46:21.530873 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.530834 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-trusted-ca-bundle\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.530873 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.530881 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-service-ca\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.531099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.530951 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rqhr\" (UniqueName: \"kubernetes.io/projected/c02b954d-6286-461f-8bfa-99a185ca55aa-kube-api-access-7rqhr\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.531099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.530997 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-oauth-config\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.531099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.531033 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-console-config\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.531197 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.531113 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-oauth-serving-cert\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.531197 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.531144 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-serving-cert\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.632181 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.632149 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-oauth-config\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.632181 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.632187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-console-config\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.632691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.632234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-oauth-serving-cert\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.632691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.632262 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-serving-cert\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.632691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.632306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-trusted-ca-bundle\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.632691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.632333 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-service-ca\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.632691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.632365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rqhr\" (UniqueName: \"kubernetes.io/projected/c02b954d-6286-461f-8bfa-99a185ca55aa-kube-api-access-7rqhr\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.633159 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.633130 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-oauth-serving-cert\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.633292 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.633270 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-service-ca\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.633354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.633295 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-console-config\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.633354 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.633295 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-trusted-ca-bundle\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.634801 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.634776 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-oauth-config\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.634901 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.634813 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-serving-cert\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.639180 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.639151 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rqhr\" (UniqueName: \"kubernetes.io/projected/c02b954d-6286-461f-8bfa-99a185ca55aa-kube-api-access-7rqhr\") pod \"console-5dff484f7c-nbn8w\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.757692 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.757613 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:21.877657 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:21.877626 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dff484f7c-nbn8w"] Jul 12 14:46:21.880721 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:46:21.880689 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02b954d_6286_461f_8bfa_99a185ca55aa.slice/crio-4b24bfc92f38215bdcbd88c35ba3ddb0ecd662e67ca291c07b3ae8d934deb1a3 WatchSource:0}: Error finding container 4b24bfc92f38215bdcbd88c35ba3ddb0ecd662e67ca291c07b3ae8d934deb1a3: Status 404 returned error can't find the container with id 4b24bfc92f38215bdcbd88c35ba3ddb0ecd662e67ca291c07b3ae8d934deb1a3 Jul 12 14:46:22.178001 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:22.177967 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:22.227844 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:22.227811 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:22.603235 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:22.603136 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dff484f7c-nbn8w" event={"ID":"c02b954d-6286-461f-8bfa-99a185ca55aa","Type":"ContainerStarted","Data":"ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa"} Jul 12 14:46:22.603235 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:22.603225 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dff484f7c-nbn8w" event={"ID":"c02b954d-6286-461f-8bfa-99a185ca55aa","Type":"ContainerStarted","Data":"4b24bfc92f38215bdcbd88c35ba3ddb0ecd662e67ca291c07b3ae8d934deb1a3"} Jul 12 14:46:22.621199 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:22.621143 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dff484f7c-nbn8w" podStartSLOduration=1.621126262 podStartE2EDuration="1.621126262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:46:22.620177829 +0000 UTC m=+330.588061095" watchObservedRunningTime="2026-07-12 14:46:22.621126262 +0000 UTC m=+330.589009515" Jul 12 14:46:22.731628 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:22.731599 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:22.833295 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:22.833262 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:31.185467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:31.185436 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:31.238582 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:31.238550 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:31.757811 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:31.757776 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:31.758002 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:31.757844 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:31.763769 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:31.763736 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:32.642396 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:32.642367 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:46:32.700693 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:32.700661 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77d65bff77-nwkqb"] Jul 12 14:46:32.810932 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:32.810897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:32.866262 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:32.866230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:32.922286 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:32.922195 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:36.658334 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:36.658299 2573 generic.go:356] "Generic (PLEG): container finished" podID="4d3fd18c-856a-42ec-a45c-2087f975bd1f" containerID="976cc703ca745334a7a3ea43acf38cc3e7abb2816c176b172876b9918d01e793" exitCode=0 Jul 12 14:46:36.658696 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:36.658353 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" event={"ID":"4d3fd18c-856a-42ec-a45c-2087f975bd1f","Type":"ContainerDied","Data":"976cc703ca745334a7a3ea43acf38cc3e7abb2816c176b172876b9918d01e793"} Jul 12 14:46:36.658696 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:36.658645 2573 scope.go:117] "RemoveContainer" containerID="976cc703ca745334a7a3ea43acf38cc3e7abb2816c176b172876b9918d01e793" Jul 12 14:46:37.624151 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.624114 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" containerName="registry" containerID="cri-o://c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916" gracePeriod=30 Jul 12 14:46:37.663636 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.663608 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-n6vd5" event={"ID":"4d3fd18c-856a-42ec-a45c-2087f975bd1f","Type":"ContainerStarted","Data":"a02cd9cd9595aefe232b07e37f926fd734eaee11cdddd15f37ceb25e84e8b600"} Jul 12 14:46:37.885253 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.885228 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:46:37.975690 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.975653 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-installation-pull-secrets\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.975690 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.975687 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-trusted-ca\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.975897 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.975707 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-bound-sa-token\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.975897 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.975723 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.975897 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.975867 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-ca-trust-extracted\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.976054 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.975925 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-image-registry-private-configuration\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.976054 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.975968 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-certificates\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.976054 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.976029 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9fx8\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-kube-api-access-q9fx8\") pod \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\" (UID: \"881b7e06-c94e-4d19-8f3d-da8a15f8efb1\") " Jul 12 14:46:37.976346 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.976190 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:46:37.976442 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.976358 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-trusted-ca\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:37.976580 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.976547 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:46:37.978386 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.978344 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:46:37.978494 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.978424 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:46:37.978494 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.978456 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:46:37.978494 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.978472 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:46:37.978661 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.978641 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-kube-api-access-q9fx8" (OuterVolumeSpecName: "kube-api-access-q9fx8") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "kube-api-access-q9fx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:46:37.984519 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:37.984494 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "881b7e06-c94e-4d19-8f3d-da8a15f8efb1" (UID: "881b7e06-c94e-4d19-8f3d-da8a15f8efb1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:46:38.077655 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.077617 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-image-registry-private-configuration\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:38.077655 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.077648 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-certificates\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:38.077655 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.077657 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9fx8\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-kube-api-access-q9fx8\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:38.077655 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.077666 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-installation-pull-secrets\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:38.077899 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.077674 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-bound-sa-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:38.077899 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.077683 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-registry-tls\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:38.077899 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.077691 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/881b7e06-c94e-4d19-8f3d-da8a15f8efb1-ca-trust-extracted\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:38.668556 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.668525 2573 generic.go:356] "Generic (PLEG): container finished" podID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" containerID="c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916" exitCode=0 Jul 12 14:46:38.668957 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.668565 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" Jul 12 14:46:38.668957 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.668608 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" event={"ID":"881b7e06-c94e-4d19-8f3d-da8a15f8efb1","Type":"ContainerDied","Data":"c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916"} Jul 12 14:46:38.668957 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.668655 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765998dfdb-wtmkw" event={"ID":"881b7e06-c94e-4d19-8f3d-da8a15f8efb1","Type":"ContainerDied","Data":"3142df3f29de4cbbe8cca5e29ee7092a7073201e8f357dcfcebeadce8553250e"} Jul 12 14:46:38.668957 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.668684 2573 scope.go:117] "RemoveContainer" containerID="c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916" Jul 12 14:46:38.677086 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.677064 2573 scope.go:117] "RemoveContainer" containerID="c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916" Jul 12 14:46:38.677349 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:46:38.677328 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916\": container with ID starting with c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916 not found: ID does not exist" containerID="c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916" Jul 12 14:46:38.677411 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.677358 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916"} err="failed to get container status \"c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916\": rpc error: code = NotFound desc = could not find container \"c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916\": container with ID starting with c6cb2a379bef7fb74fb1793d84c07c8f0469ca9a9a7a883e22b6240502299916 not found: ID does not exist" Jul 12 14:46:38.683879 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.683852 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-765998dfdb-wtmkw"] Jul 12 14:46:38.686968 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:38.686950 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-765998dfdb-wtmkw"] Jul 12 14:46:40.575514 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:40.575482 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" path="/var/lib/kubelet/pods/881b7e06-c94e-4d19-8f3d-da8a15f8efb1/volumes" Jul 12 14:46:41.782965 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:41.782933 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:41.883751 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:41.883718 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:46:57.727602 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:57.727565 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77d65bff77-nwkqb" podUID="b648a5d2-d158-4008-9020-f635ff751627" containerName="console" containerID="cri-o://d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9" gracePeriod=15 Jul 12 14:46:57.974634 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:57.974609 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:58.045295 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045207 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-service-ca\") pod \"b648a5d2-d158-4008-9020-f635ff751627\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " Jul 12 14:46:58.045295 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045253 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52kvj\" (UniqueName: \"kubernetes.io/projected/b648a5d2-d158-4008-9020-f635ff751627-kube-api-access-52kvj\") pod \"b648a5d2-d158-4008-9020-f635ff751627\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " Jul 12 14:46:58.045295 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045286 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-oauth-config\") pod \"b648a5d2-d158-4008-9020-f635ff751627\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " Jul 12 14:46:58.045564 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045302 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-console-config\") pod \"b648a5d2-d158-4008-9020-f635ff751627\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " Jul 12 14:46:58.045564 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045473 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-oauth-serving-cert\") pod \"b648a5d2-d158-4008-9020-f635ff751627\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " Jul 12 14:46:58.045564 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045537 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-serving-cert\") pod \"b648a5d2-d158-4008-9020-f635ff751627\" (UID: \"b648a5d2-d158-4008-9020-f635ff751627\") " Jul 12 14:46:58.045805 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045747 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-service-ca" (OuterVolumeSpecName: "service-ca") pod "b648a5d2-d158-4008-9020-f635ff751627" (UID: "b648a5d2-d158-4008-9020-f635ff751627"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:46:58.045805 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045754 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-console-config" (OuterVolumeSpecName: "console-config") pod "b648a5d2-d158-4008-9020-f635ff751627" (UID: "b648a5d2-d158-4008-9020-f635ff751627"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:46:58.045961 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.045807 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b648a5d2-d158-4008-9020-f635ff751627" (UID: "b648a5d2-d158-4008-9020-f635ff751627"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:46:58.047560 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.047529 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b648a5d2-d158-4008-9020-f635ff751627" (UID: "b648a5d2-d158-4008-9020-f635ff751627"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:46:58.047647 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.047579 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b648a5d2-d158-4008-9020-f635ff751627" (UID: "b648a5d2-d158-4008-9020-f635ff751627"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:46:58.047647 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.047588 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b648a5d2-d158-4008-9020-f635ff751627-kube-api-access-52kvj" (OuterVolumeSpecName: "kube-api-access-52kvj") pod "b648a5d2-d158-4008-9020-f635ff751627" (UID: "b648a5d2-d158-4008-9020-f635ff751627"). InnerVolumeSpecName "kube-api-access-52kvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:46:58.147097 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.147056 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-oauth-config\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:58.147097 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.147086 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-console-config\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:58.147097 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.147097 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-oauth-serving-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:58.147097 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.147111 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b648a5d2-d158-4008-9020-f635ff751627-console-serving-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:58.147378 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.147129 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b648a5d2-d158-4008-9020-f635ff751627-service-ca\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:58.147378 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.147138 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52kvj\" (UniqueName: \"kubernetes.io/projected/b648a5d2-d158-4008-9020-f635ff751627-kube-api-access-52kvj\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:46:58.747986 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.747952 2573 generic.go:356] "Generic (PLEG): container finished" podID="b648a5d2-d158-4008-9020-f635ff751627" containerID="d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9" exitCode=0 Jul 12 14:46:58.748414 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.748027 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d65bff77-nwkqb" Jul 12 14:46:58.748414 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.748035 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d65bff77-nwkqb" event={"ID":"b648a5d2-d158-4008-9020-f635ff751627","Type":"ContainerDied","Data":"d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9"} Jul 12 14:46:58.748414 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.748118 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d65bff77-nwkqb" event={"ID":"b648a5d2-d158-4008-9020-f635ff751627","Type":"ContainerDied","Data":"d4509a3c9e29a06963d8c6f549619269b7b8817325dbcade871fa30775f6348e"} Jul 12 14:46:58.748414 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.748144 2573 scope.go:117] "RemoveContainer" containerID="d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9" Jul 12 14:46:58.756006 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.755985 2573 scope.go:117] "RemoveContainer" containerID="d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9" Jul 12 14:46:58.756262 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:46:58.756244 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9\": container with ID starting with d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9 not found: ID does not exist" containerID="d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9" Jul 12 14:46:58.756316 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.756271 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9"} err="failed to get container status \"d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9\": rpc error: code = NotFound desc = could not find container \"d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9\": container with ID starting with d6ca2e0f41751a6424c39042758d2ac2e356e875d26fe6b9f9c22cfa114cebc9 not found: ID does not exist" Jul 12 14:46:58.762255 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.762231 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77d65bff77-nwkqb"] Jul 12 14:46:58.765989 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:46:58.765968 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77d65bff77-nwkqb"] Jul 12 14:47:00.575107 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:00.575072 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b648a5d2-d158-4008-9020-f635ff751627" path="/var/lib/kubelet/pods/b648a5d2-d158-4008-9020-f635ff751627/volumes" Jul 12 14:47:06.615952 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:06.615915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:47:06.618202 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:06.618180 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab38523-f4fb-4fb3-9111-4dd8070e71de-metrics-certs\") pod \"network-metrics-daemon-fjw4t\" (UID: \"9ab38523-f4fb-4fb3-9111-4dd8070e71de\") " pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:47:06.874579 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:06.874555 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6tgvw\"" Jul 12 14:47:06.883202 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:06.883178 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjw4t" Jul 12 14:47:07.004293 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:07.004208 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjw4t"] Jul 12 14:47:07.007066 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:47:07.007001 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab38523_f4fb_4fb3_9111_4dd8070e71de.slice/crio-fd259802325c839b1442f462281e2a075ad4c78a0154824201b0c8cab9adf22b WatchSource:0}: Error finding container fd259802325c839b1442f462281e2a075ad4c78a0154824201b0c8cab9adf22b: Status 404 returned error can't find the container with id fd259802325c839b1442f462281e2a075ad4c78a0154824201b0c8cab9adf22b Jul 12 14:47:07.786581 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:07.786533 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjw4t" event={"ID":"9ab38523-f4fb-4fb3-9111-4dd8070e71de","Type":"ContainerStarted","Data":"fd259802325c839b1442f462281e2a075ad4c78a0154824201b0c8cab9adf22b"} Jul 12 14:47:08.792132 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:08.792098 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjw4t" event={"ID":"9ab38523-f4fb-4fb3-9111-4dd8070e71de","Type":"ContainerStarted","Data":"a8cc30de1470d2ea6ec9396e84f65025793dca7aa2a7cb8192dafd2caf58cd75"} Jul 12 14:47:08.792610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:08.792156 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjw4t" event={"ID":"9ab38523-f4fb-4fb3-9111-4dd8070e71de","Type":"ContainerStarted","Data":"9a399968c8714df8a32b65399029dac6506279ccdce1a75f2fd9be2bdbd4ea0a"} Jul 12 14:47:08.807792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:08.807694 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fjw4t" podStartSLOduration=375.764123986 podStartE2EDuration="6m16.807673881s" totalImagesPullingTime="1.043549895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:47:08.806798058 +0000 UTC m=+376.774681321" watchObservedRunningTime="2026-07-12 14:47:08.807673881 +0000 UTC m=+376.775557135" Jul 12 14:47:08.935652 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:08.935622 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:08.992814 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:08.992784 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:27.820694 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:27.820660 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:27.922068 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:27.922009 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:29.045938 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:29.045906 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:29.098177 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:29.098148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:30.702433 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.702403 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c5d6dc4d7-2p55w"] Jul 12 14:47:30.702900 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.702779 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" containerName="registry" Jul 12 14:47:30.702900 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.702795 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" containerName="registry" Jul 12 14:47:30.702900 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.702808 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b648a5d2-d158-4008-9020-f635ff751627" containerName="console" Jul 12 14:47:30.702900 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.702817 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b648a5d2-d158-4008-9020-f635ff751627" containerName="console" Jul 12 14:47:30.703135 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.702904 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="881b7e06-c94e-4d19-8f3d-da8a15f8efb1" containerName="registry" Jul 12 14:47:30.703135 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.702916 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b648a5d2-d158-4008-9020-f635ff751627" containerName="console" Jul 12 14:47:30.706735 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.706500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.715585 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.715559 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c5d6dc4d7-2p55w"] Jul 12 14:47:30.763308 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.763246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:30.823656 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.823614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7wp5\" (UniqueName: \"kubernetes.io/projected/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-kube-api-access-f7wp5\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.823656 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.823659 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-oauth-serving-cert\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.823895 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.823687 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-trusted-ca-bundle\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.823895 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.823791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-service-ca\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.823895 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.823835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-oauth-config\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.823895 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.823874 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-serving-cert\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.823895 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.823894 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-config\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.863151 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.863108 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:30.924608 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.924565 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-service-ca\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.924608 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.924608 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-oauth-config\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.924857 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.924645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-serving-cert\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.924857 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.924662 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-config\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.924857 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.924681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7wp5\" (UniqueName: \"kubernetes.io/projected/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-kube-api-access-f7wp5\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.924857 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.924703 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-oauth-serving-cert\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.924857 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.924742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-trusted-ca-bundle\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.925448 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.925418 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-service-ca\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.925566 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.925517 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-oauth-serving-cert\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.925566 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.925517 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-config\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.925677 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.925651 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-trusted-ca-bundle\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.927147 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.927118 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-serving-cert\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.927230 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.927168 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-oauth-config\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.935509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.935486 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7wp5\" (UniqueName: \"kubernetes.io/projected/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-kube-api-access-f7wp5\") pod \"console-c5d6dc4d7-2p55w\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:30.963314 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:30.963224 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:31.017241 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.017205 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:31.146954 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.146921 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c5d6dc4d7-2p55w"] Jul 12 14:47:31.150784 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:47:31.150758 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d4e9f0_6886_4687_897d_b6cb1646d4fe.slice/crio-4221a00a9881d596c92bc06cfb957c704a989beb69b2f2c345ca10e71a0a6ff3 WatchSource:0}: Error finding container 4221a00a9881d596c92bc06cfb957c704a989beb69b2f2c345ca10e71a0a6ff3: Status 404 returned error can't find the container with id 4221a00a9881d596c92bc06cfb957c704a989beb69b2f2c345ca10e71a0a6ff3 Jul 12 14:47:31.500766 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.500729 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:31.602603 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.602572 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:31.703632 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.703600 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:31.877582 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.877549 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c5d6dc4d7-2p55w" event={"ID":"e6d4e9f0-6886-4687-897d-b6cb1646d4fe","Type":"ContainerStarted","Data":"bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab"} Jul 12 14:47:31.877772 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.877606 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c5d6dc4d7-2p55w" event={"ID":"e6d4e9f0-6886-4687-897d-b6cb1646d4fe","Type":"ContainerStarted","Data":"4221a00a9881d596c92bc06cfb957c704a989beb69b2f2c345ca10e71a0a6ff3"} Jul 12 14:47:31.907505 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:31.907442 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c5d6dc4d7-2p55w" podStartSLOduration=1.907425602 podStartE2EDuration="1.907425602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:47:31.906785558 +0000 UTC m=+399.874668811" watchObservedRunningTime="2026-07-12 14:47:31.907425602 +0000 UTC m=+399.875308933" Jul 12 14:47:37.397072 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:47:37.397007 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" podUID="10e3433a-1692-4ea9-8e91-be8d0739b2d6" Jul 12 14:47:37.898924 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:37.898896 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:47:39.496544 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.496495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:47:39.498812 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.498790 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/10e3433a-1692-4ea9-8e91-be8d0739b2d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-pstf2\" (UID: \"10e3433a-1692-4ea9-8e91-be8d0739b2d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:47:39.701851 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.701824 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8rjvj\"" Jul 12 14:47:39.709465 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.709443 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" Jul 12 14:47:39.802553 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.802520 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:39.834417 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.834383 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-pstf2"] Jul 12 14:47:39.837087 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:47:39.837060 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e3433a_1692_4ea9_8e91_be8d0739b2d6.slice/crio-1730969dea3232e51a38c84846d4786d5a284eb9a567c3ee82d225a9d8705c82 WatchSource:0}: Error finding container 1730969dea3232e51a38c84846d4786d5a284eb9a567c3ee82d225a9d8705c82: Status 404 returned error can't find the container with id 1730969dea3232e51a38c84846d4786d5a284eb9a567c3ee82d225a9d8705c82 Jul 12 14:47:39.901105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.901073 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:39.907079 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:39.907043 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" event={"ID":"10e3433a-1692-4ea9-8e91-be8d0739b2d6","Type":"ContainerStarted","Data":"1730969dea3232e51a38c84846d4786d5a284eb9a567c3ee82d225a9d8705c82"} Jul 12 14:47:41.018102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:41.018075 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:41.018450 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:41.018141 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:41.023985 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:41.023964 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:41.917473 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:41.917436 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" event={"ID":"10e3433a-1692-4ea9-8e91-be8d0739b2d6","Type":"ContainerStarted","Data":"00614bed805a7a839aa6b61ec32dc2bb725995f914079e04b6bc1e04eec8916e"} Jul 12 14:47:41.921126 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:41.921095 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:47:41.932071 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:41.932004 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-pstf2" podStartSLOduration=380.763455583 podStartE2EDuration="6m21.931990037s" totalImagesPullingTime="1.168534454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:47:41.931540303 +0000 UTC m=+409.899423547" watchObservedRunningTime="2026-07-12 14:47:41.931990037 +0000 UTC m=+409.899873328" Jul 12 14:47:41.977602 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:41.977571 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dff484f7c-nbn8w"] Jul 12 14:47:42.038691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:42.038658 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:42.140061 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:42.140007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:42.239869 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:42.239781 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:54.103156 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:54.103107 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:54.154886 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:54.154853 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:56.797596 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:56.797557 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:47:56.899065 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:47:56.899036 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:07.001914 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.001877 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dff484f7c-nbn8w" podUID="c02b954d-6286-461f-8bfa-99a185ca55aa" containerName="console" containerID="cri-o://ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa" gracePeriod=15 Jul 12 14:48:07.242608 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.242581 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:48:07.305691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.305614 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-oauth-config\") pod \"c02b954d-6286-461f-8bfa-99a185ca55aa\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " Jul 12 14:48:07.305691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.305653 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-console-config\") pod \"c02b954d-6286-461f-8bfa-99a185ca55aa\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " Jul 12 14:48:07.305691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.305676 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rqhr\" (UniqueName: \"kubernetes.io/projected/c02b954d-6286-461f-8bfa-99a185ca55aa-kube-api-access-7rqhr\") pod \"c02b954d-6286-461f-8bfa-99a185ca55aa\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " Jul 12 14:48:07.305956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.305746 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-oauth-serving-cert\") pod \"c02b954d-6286-461f-8bfa-99a185ca55aa\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " Jul 12 14:48:07.305956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.305778 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-serving-cert\") pod \"c02b954d-6286-461f-8bfa-99a185ca55aa\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " Jul 12 14:48:07.305956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.305825 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-trusted-ca-bundle\") pod \"c02b954d-6286-461f-8bfa-99a185ca55aa\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " Jul 12 14:48:07.305956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.305844 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-service-ca\") pod \"c02b954d-6286-461f-8bfa-99a185ca55aa\" (UID: \"c02b954d-6286-461f-8bfa-99a185ca55aa\") " Jul 12 14:48:07.306220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.306191 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-console-config" (OuterVolumeSpecName: "console-config") pod "c02b954d-6286-461f-8bfa-99a185ca55aa" (UID: "c02b954d-6286-461f-8bfa-99a185ca55aa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:48:07.306273 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.306191 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c02b954d-6286-461f-8bfa-99a185ca55aa" (UID: "c02b954d-6286-461f-8bfa-99a185ca55aa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:48:07.306329 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.306285 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-service-ca" (OuterVolumeSpecName: "service-ca") pod "c02b954d-6286-461f-8bfa-99a185ca55aa" (UID: "c02b954d-6286-461f-8bfa-99a185ca55aa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:48:07.306329 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.306299 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c02b954d-6286-461f-8bfa-99a185ca55aa" (UID: "c02b954d-6286-461f-8bfa-99a185ca55aa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:48:07.307789 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.307769 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c02b954d-6286-461f-8bfa-99a185ca55aa-kube-api-access-7rqhr" (OuterVolumeSpecName: "kube-api-access-7rqhr") pod "c02b954d-6286-461f-8bfa-99a185ca55aa" (UID: "c02b954d-6286-461f-8bfa-99a185ca55aa"). InnerVolumeSpecName "kube-api-access-7rqhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:48:07.308263 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.308238 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c02b954d-6286-461f-8bfa-99a185ca55aa" (UID: "c02b954d-6286-461f-8bfa-99a185ca55aa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:48:07.308263 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.308250 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c02b954d-6286-461f-8bfa-99a185ca55aa" (UID: "c02b954d-6286-461f-8bfa-99a185ca55aa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:48:07.406695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.406647 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-trusted-ca-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:07.406695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.406691 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-service-ca\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:07.406695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.406701 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-oauth-config\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:07.406695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.406709 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-console-config\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:07.406985 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.406719 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rqhr\" (UniqueName: \"kubernetes.io/projected/c02b954d-6286-461f-8bfa-99a185ca55aa-kube-api-access-7rqhr\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:07.406985 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.406728 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02b954d-6286-461f-8bfa-99a185ca55aa-oauth-serving-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:07.406985 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:07.406736 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02b954d-6286-461f-8bfa-99a185ca55aa-console-serving-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:08.010992 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.010957 2573 generic.go:356] "Generic (PLEG): container finished" podID="c02b954d-6286-461f-8bfa-99a185ca55aa" containerID="ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa" exitCode=0 Jul 12 14:48:08.011397 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.011000 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dff484f7c-nbn8w" Jul 12 14:48:08.011397 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.011023 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dff484f7c-nbn8w" event={"ID":"c02b954d-6286-461f-8bfa-99a185ca55aa","Type":"ContainerDied","Data":"ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa"} Jul 12 14:48:08.011397 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.011091 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dff484f7c-nbn8w" event={"ID":"c02b954d-6286-461f-8bfa-99a185ca55aa","Type":"ContainerDied","Data":"4b24bfc92f38215bdcbd88c35ba3ddb0ecd662e67ca291c07b3ae8d934deb1a3"} Jul 12 14:48:08.011397 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.011115 2573 scope.go:117] "RemoveContainer" containerID="ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa" Jul 12 14:48:08.019179 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.019164 2573 scope.go:117] "RemoveContainer" containerID="ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa" Jul 12 14:48:08.019415 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:48:08.019396 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa\": container with ID starting with ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa not found: ID does not exist" containerID="ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa" Jul 12 14:48:08.019462 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.019422 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa"} err="failed to get container status \"ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa\": rpc error: code = NotFound desc = could not find container \"ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa\": container with ID starting with ab919613316bee895f95731c936046c55a17049a0f41059478f7396dbe338eaa not found: ID does not exist" Jul 12 14:48:08.030718 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.030693 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dff484f7c-nbn8w"] Jul 12 14:48:08.034851 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.034827 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dff484f7c-nbn8w"] Jul 12 14:48:08.579898 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:08.579866 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c02b954d-6286-461f-8bfa-99a185ca55aa" path="/var/lib/kubelet/pods/c02b954d-6286-461f-8bfa-99a185ca55aa/volumes" Jul 12 14:48:19.447570 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:19.447487 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:19.548235 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:19.548164 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:20.080330 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.080302 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4"] Jul 12 14:48:20.080569 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.080558 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c02b954d-6286-461f-8bfa-99a185ca55aa" containerName="console" Jul 12 14:48:20.080610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.080570 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02b954d-6286-461f-8bfa-99a185ca55aa" containerName="console" Jul 12 14:48:20.080642 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.080624 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c02b954d-6286-461f-8bfa-99a185ca55aa" containerName="console" Jul 12 14:48:20.083452 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.083436 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.085568 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.085539 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 14:48:20.085568 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.085568 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 14:48:20.085762 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.085634 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qpjdm\"" Jul 12 14:48:20.092146 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.092097 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4"] Jul 12 14:48:20.205519 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.205481 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.205705 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.205562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.205705 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.205580 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxh8j\" (UniqueName: \"kubernetes.io/projected/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-kube-api-access-cxh8j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.306099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.306063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.306099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.306099 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxh8j\" (UniqueName: \"kubernetes.io/projected/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-kube-api-access-cxh8j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.306302 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.306125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.306438 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.306418 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.306480 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.306440 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.314219 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.314197 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxh8j\" (UniqueName: \"kubernetes.io/projected/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-kube-api-access-cxh8j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.392867 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.392832 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:20.517031 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:20.516982 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4"] Jul 12 14:48:20.522406 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:48:20.520376 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c3a6da_5e6a_4414_8b7c_382cfff91ff7.slice/crio-113f06c823486f1c306b2a20dabd3d34d2d345759a1134a7f0aa72227d93ea0e WatchSource:0}: Error finding container 113f06c823486f1c306b2a20dabd3d34d2d345759a1134a7f0aa72227d93ea0e: Status 404 returned error can't find the container with id 113f06c823486f1c306b2a20dabd3d34d2d345759a1134a7f0aa72227d93ea0e Jul 12 14:48:21.060674 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:21.060644 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" event={"ID":"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7","Type":"ContainerStarted","Data":"113f06c823486f1c306b2a20dabd3d34d2d345759a1134a7f0aa72227d93ea0e"} Jul 12 14:48:27.088817 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:27.088303 2573 generic.go:356] "Generic (PLEG): container finished" podID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerID="6a9d43e26f9a54569f0b6c02bbc566ce7c9aa36af5ece2b562dbf54e8ea223f4" exitCode=0 Jul 12 14:48:27.088817 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:27.088349 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" event={"ID":"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7","Type":"ContainerDied","Data":"6a9d43e26f9a54569f0b6c02bbc566ce7c9aa36af5ece2b562dbf54e8ea223f4"} Jul 12 14:48:29.096736 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:29.096703 2573 generic.go:356] "Generic (PLEG): container finished" podID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerID="50033813f842713ea8ac28f7d1db8d809027968a4a5a6a947a91ff28d12c1b85" exitCode=0 Jul 12 14:48:29.097121 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:29.096757 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" event={"ID":"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7","Type":"ContainerDied","Data":"50033813f842713ea8ac28f7d1db8d809027968a4a5a6a947a91ff28d12c1b85"} Jul 12 14:48:37.131893 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:37.131855 2573 generic.go:356] "Generic (PLEG): container finished" podID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerID="de8951c0174ad14a8831d35102619aed039c44c04891f7f18914c03105b9908b" exitCode=0 Jul 12 14:48:37.132333 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:37.131918 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" event={"ID":"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7","Type":"ContainerDied","Data":"de8951c0174ad14a8831d35102619aed039c44c04891f7f18914c03105b9908b"} Jul 12 14:48:38.259188 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.259166 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:38.360120 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.360087 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-util\") pod \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " Jul 12 14:48:38.360299 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.360144 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxh8j\" (UniqueName: \"kubernetes.io/projected/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-kube-api-access-cxh8j\") pod \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " Jul 12 14:48:38.360299 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.360176 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-bundle\") pod \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\" (UID: \"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7\") " Jul 12 14:48:38.360920 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.360882 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-bundle" (OuterVolumeSpecName: "bundle") pod "c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" (UID: "c1c3a6da-5e6a-4414-8b7c-382cfff91ff7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:48:38.362466 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.362448 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-kube-api-access-cxh8j" (OuterVolumeSpecName: "kube-api-access-cxh8j") pod "c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" (UID: "c1c3a6da-5e6a-4414-8b7c-382cfff91ff7"). InnerVolumeSpecName "kube-api-access-cxh8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:48:38.364063 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.364044 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-util" (OuterVolumeSpecName: "util") pod "c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" (UID: "c1c3a6da-5e6a-4414-8b7c-382cfff91ff7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:48:38.461480 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.461386 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-util\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:38.461480 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.461417 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxh8j\" (UniqueName: \"kubernetes.io/projected/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-kube-api-access-cxh8j\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:38.461480 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:38.461431 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1c3a6da-5e6a-4414-8b7c-382cfff91ff7-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:39.140849 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:39.140821 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" Jul 12 14:48:39.141053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:39.140816 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n54k4" event={"ID":"c1c3a6da-5e6a-4414-8b7c-382cfff91ff7","Type":"ContainerDied","Data":"113f06c823486f1c306b2a20dabd3d34d2d345759a1134a7f0aa72227d93ea0e"} Jul 12 14:48:39.141053 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:39.140951 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="113f06c823486f1c306b2a20dabd3d34d2d345759a1134a7f0aa72227d93ea0e" Jul 12 14:48:40.848408 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:40.848369 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:42.242333 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.242294 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:42.799691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.799659 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf"] Jul 12 14:48:42.799939 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.799929 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerName="extract" Jul 12 14:48:42.799984 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.799942 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerName="extract" Jul 12 14:48:42.799984 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.799953 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerName="util" Jul 12 14:48:42.799984 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.799958 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerName="util" Jul 12 14:48:42.799984 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.799966 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerName="pull" Jul 12 14:48:42.799984 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.799971 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerName="pull" Jul 12 14:48:42.800155 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.800029 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1c3a6da-5e6a-4414-8b7c-382cfff91ff7" containerName="extract" Jul 12 14:48:42.803441 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.803425 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:42.805547 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.805520 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 12 14:48:42.805547 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.805538 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zvhmm\"" Jul 12 14:48:42.805716 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.805565 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 12 14:48:42.817399 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.817377 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf"] Jul 12 14:48:42.898213 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.898182 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f94df9c-e086-43ab-aa3d-74b9d1707054-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6t9mf\" (UID: \"8f94df9c-e086-43ab-aa3d-74b9d1707054\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:42.898369 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.898224 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwfsq\" (UniqueName: \"kubernetes.io/projected/8f94df9c-e086-43ab-aa3d-74b9d1707054-kube-api-access-nwfsq\") pod \"cert-manager-operator-controller-manager-55c444bf86-6t9mf\" (UID: \"8f94df9c-e086-43ab-aa3d-74b9d1707054\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:42.999537 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.999491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f94df9c-e086-43ab-aa3d-74b9d1707054-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6t9mf\" (UID: \"8f94df9c-e086-43ab-aa3d-74b9d1707054\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:42.999715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.999564 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwfsq\" (UniqueName: \"kubernetes.io/projected/8f94df9c-e086-43ab-aa3d-74b9d1707054-kube-api-access-nwfsq\") pod \"cert-manager-operator-controller-manager-55c444bf86-6t9mf\" (UID: \"8f94df9c-e086-43ab-aa3d-74b9d1707054\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:42.999861 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:42.999839 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f94df9c-e086-43ab-aa3d-74b9d1707054-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6t9mf\" (UID: \"8f94df9c-e086-43ab-aa3d-74b9d1707054\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:43.010994 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:43.010974 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwfsq\" (UniqueName: \"kubernetes.io/projected/8f94df9c-e086-43ab-aa3d-74b9d1707054-kube-api-access-nwfsq\") pod \"cert-manager-operator-controller-manager-55c444bf86-6t9mf\" (UID: \"8f94df9c-e086-43ab-aa3d-74b9d1707054\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:43.112369 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:43.112270 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" Jul 12 14:48:43.239858 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:43.239827 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf"] Jul 12 14:48:43.243117 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:48:43.243085 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f94df9c_e086_43ab_aa3d_74b9d1707054.slice/crio-096eff4890e525d3c9c4ac9c6136f7215599c5091526f02cf7c8ccc3de417a46 WatchSource:0}: Error finding container 096eff4890e525d3c9c4ac9c6136f7215599c5091526f02cf7c8ccc3de417a46: Status 404 returned error can't find the container with id 096eff4890e525d3c9c4ac9c6136f7215599c5091526f02cf7c8ccc3de417a46 Jul 12 14:48:44.167170 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:44.167122 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" event={"ID":"8f94df9c-e086-43ab-aa3d-74b9d1707054","Type":"ContainerStarted","Data":"096eff4890e525d3c9c4ac9c6136f7215599c5091526f02cf7c8ccc3de417a46"} Jul 12 14:48:44.468753 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:44.468652 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:44.568691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:44.568653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:45.176038 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:45.175977 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" event={"ID":"8f94df9c-e086-43ab-aa3d-74b9d1707054","Type":"ContainerStarted","Data":"121d7cf44f872294460c7b4dbfc6853d35f826df02eba87da06b6b6c5dae9953"} Jul 12 14:48:45.200377 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:45.200327 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6t9mf" podStartSLOduration=1.794971967 podStartE2EDuration="3.200308638s" totalImagesPullingTime="1.405336671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:48:45.199467566 +0000 UTC m=+473.167350844" watchObservedRunningTime="2026-07-12 14:48:45.200308638 +0000 UTC m=+473.168191890" Jul 12 14:48:45.537197 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:45.537114 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:45.646758 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:45.646724 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:46.420892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.420854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:46.529075 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.529042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:46.575894 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.575861 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc"] Jul 12 14:48:46.578257 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.578236 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.580440 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.580420 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 14:48:46.580552 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.580511 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qpjdm\"" Jul 12 14:48:46.581038 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.581027 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 14:48:46.589893 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.589859 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc"] Jul 12 14:48:46.728370 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.728284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwwkq\" (UniqueName: \"kubernetes.io/projected/abf6007b-69f2-4aeb-9b52-6d7136befe0e-kube-api-access-fwwkq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.728370 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.728321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.728370 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.728352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.828913 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.828853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwwkq\" (UniqueName: \"kubernetes.io/projected/abf6007b-69f2-4aeb-9b52-6d7136befe0e-kube-api-access-fwwkq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.828913 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.828916 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.829175 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.828980 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.829363 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.829343 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.829402 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.829381 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.839089 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.839056 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwwkq\" (UniqueName: \"kubernetes.io/projected/abf6007b-69f2-4aeb-9b52-6d7136befe0e-kube-api-access-fwwkq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:46.888183 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:46.888151 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:47.009466 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:47.009440 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc"] Jul 12 14:48:47.011470 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:48:47.011442 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf6007b_69f2_4aeb_9b52_6d7136befe0e.slice/crio-2d711fc476379cd45c2fc0deac38c00e93b3d515e2c5d81aceef4922a32c9fc2 WatchSource:0}: Error finding container 2d711fc476379cd45c2fc0deac38c00e93b3d515e2c5d81aceef4922a32c9fc2: Status 404 returned error can't find the container with id 2d711fc476379cd45c2fc0deac38c00e93b3d515e2c5d81aceef4922a32c9fc2 Jul 12 14:48:47.185204 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:47.185172 2573 generic.go:356] "Generic (PLEG): container finished" podID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerID="118669bb788be40df95e2dbe55d13d1a266a379d0aa126de8673f8ad42b94876" exitCode=0 Jul 12 14:48:47.185204 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:47.185206 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" event={"ID":"abf6007b-69f2-4aeb-9b52-6d7136befe0e","Type":"ContainerDied","Data":"118669bb788be40df95e2dbe55d13d1a266a379d0aa126de8673f8ad42b94876"} Jul 12 14:48:47.185422 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:47.185241 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" event={"ID":"abf6007b-69f2-4aeb-9b52-6d7136befe0e","Type":"ContainerStarted","Data":"2d711fc476379cd45c2fc0deac38c00e93b3d515e2c5d81aceef4922a32c9fc2"} Jul 12 14:48:48.629231 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:48.629196 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:48.816178 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:48.816143 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:49.199956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.199922 2573 generic.go:356] "Generic (PLEG): container finished" podID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerID="7f393f39ad9ca289d2094559f268a758b5a9a8ef05e4e73f053ddd7002b5105a" exitCode=0 Jul 12 14:48:49.200157 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.199956 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" event={"ID":"abf6007b-69f2-4aeb-9b52-6d7136befe0e","Type":"ContainerDied","Data":"7f393f39ad9ca289d2094559f268a758b5a9a8ef05e4e73f053ddd7002b5105a"} Jul 12 14:48:49.313756 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.313681 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl"] Jul 12 14:48:49.315969 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.315949 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.318001 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.317979 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 14:48:49.318156 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.317981 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-76rg5\"" Jul 12 14:48:49.318156 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.318005 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 14:48:49.324563 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.324541 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl"] Jul 12 14:48:49.452483 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.452445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd7lk\" (UniqueName: \"kubernetes.io/projected/19bc046f-8f01-4e86-906f-8c8c7a3d632a-kube-api-access-zd7lk\") pod \"cert-manager-cainjector-559cdd8c87-vkmzl\" (UID: \"19bc046f-8f01-4e86-906f-8c8c7a3d632a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.452659 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.452512 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19bc046f-8f01-4e86-906f-8c8c7a3d632a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vkmzl\" (UID: \"19bc046f-8f01-4e86-906f-8c8c7a3d632a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.552883 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.552849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd7lk\" (UniqueName: \"kubernetes.io/projected/19bc046f-8f01-4e86-906f-8c8c7a3d632a-kube-api-access-zd7lk\") pod \"cert-manager-cainjector-559cdd8c87-vkmzl\" (UID: \"19bc046f-8f01-4e86-906f-8c8c7a3d632a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.553077 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.552915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19bc046f-8f01-4e86-906f-8c8c7a3d632a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vkmzl\" (UID: \"19bc046f-8f01-4e86-906f-8c8c7a3d632a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.560176 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.560147 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19bc046f-8f01-4e86-906f-8c8c7a3d632a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vkmzl\" (UID: \"19bc046f-8f01-4e86-906f-8c8c7a3d632a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.560288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.560267 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd7lk\" (UniqueName: \"kubernetes.io/projected/19bc046f-8f01-4e86-906f-8c8c7a3d632a-kube-api-access-zd7lk\") pod \"cert-manager-cainjector-559cdd8c87-vkmzl\" (UID: \"19bc046f-8f01-4e86-906f-8c8c7a3d632a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.669684 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.669650 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" Jul 12 14:48:49.785622 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:49.785590 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl"] Jul 12 14:48:49.788390 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:48:49.788367 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19bc046f_8f01_4e86_906f_8c8c7a3d632a.slice/crio-76fff45bd9b5368790fcf24a8f217b1d8e55c8fa0f7a3c5241f6871324d5e58e WatchSource:0}: Error finding container 76fff45bd9b5368790fcf24a8f217b1d8e55c8fa0f7a3c5241f6871324d5e58e: Status 404 returned error can't find the container with id 76fff45bd9b5368790fcf24a8f217b1d8e55c8fa0f7a3c5241f6871324d5e58e Jul 12 14:48:50.205505 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:50.205470 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" event={"ID":"19bc046f-8f01-4e86-906f-8c8c7a3d632a","Type":"ContainerStarted","Data":"76fff45bd9b5368790fcf24a8f217b1d8e55c8fa0f7a3c5241f6871324d5e58e"} Jul 12 14:48:50.207456 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:50.207434 2573 generic.go:356] "Generic (PLEG): container finished" podID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerID="e6cf8120c680340f2c3a92d9ebfe8fdfd5d087932280ef2cc7badf4de25adc67" exitCode=0 Jul 12 14:48:50.207571 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:50.207463 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" event={"ID":"abf6007b-69f2-4aeb-9b52-6d7136befe0e","Type":"ContainerDied","Data":"e6cf8120c680340f2c3a92d9ebfe8fdfd5d087932280ef2cc7badf4de25adc67"} Jul 12 14:48:51.356878 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.356846 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:51.467956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.467880 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-util\") pod \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " Jul 12 14:48:51.467956 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.467920 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-bundle\") pod \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " Jul 12 14:48:51.468184 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.467988 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwwkq\" (UniqueName: \"kubernetes.io/projected/abf6007b-69f2-4aeb-9b52-6d7136befe0e-kube-api-access-fwwkq\") pod \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\" (UID: \"abf6007b-69f2-4aeb-9b52-6d7136befe0e\") " Jul 12 14:48:51.468410 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.468380 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-bundle" (OuterVolumeSpecName: "bundle") pod "abf6007b-69f2-4aeb-9b52-6d7136befe0e" (UID: "abf6007b-69f2-4aeb-9b52-6d7136befe0e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:48:51.470589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.470560 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf6007b-69f2-4aeb-9b52-6d7136befe0e-kube-api-access-fwwkq" (OuterVolumeSpecName: "kube-api-access-fwwkq") pod "abf6007b-69f2-4aeb-9b52-6d7136befe0e" (UID: "abf6007b-69f2-4aeb-9b52-6d7136befe0e"). InnerVolumeSpecName "kube-api-access-fwwkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:48:51.474219 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.474193 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-util" (OuterVolumeSpecName: "util") pod "abf6007b-69f2-4aeb-9b52-6d7136befe0e" (UID: "abf6007b-69f2-4aeb-9b52-6d7136befe0e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:48:51.532043 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.531994 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:51.569213 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.569173 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-util\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:51.569213 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.569214 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf6007b-69f2-4aeb-9b52-6d7136befe0e-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:51.569402 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.569229 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwwkq\" (UniqueName: \"kubernetes.io/projected/abf6007b-69f2-4aeb-9b52-6d7136befe0e-kube-api-access-fwwkq\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:48:51.583763 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:51.583729 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:52.218752 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:52.218721 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" event={"ID":"abf6007b-69f2-4aeb-9b52-6d7136befe0e","Type":"ContainerDied","Data":"2d711fc476379cd45c2fc0deac38c00e93b3d515e2c5d81aceef4922a32c9fc2"} Jul 12 14:48:52.218752 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:52.218767 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d711fc476379cd45c2fc0deac38c00e93b3d515e2c5d81aceef4922a32c9fc2" Jul 12 14:48:52.218955 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:52.218773 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqkkpc" Jul 12 14:48:53.224267 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.224229 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" event={"ID":"19bc046f-8f01-4e86-906f-8c8c7a3d632a","Type":"ContainerStarted","Data":"8c7755e99762a2afb44083b59484b948cde376f7ce0774d1f6646d259ab3a763"} Jul 12 14:48:53.242782 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.242732 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vkmzl" podStartSLOduration=1.692979231 podStartE2EDuration="4.24271619s" totalImagesPullingTime="2.549736959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:48:53.241948901 +0000 UTC m=+481.209832153" watchObservedRunningTime="2026-07-12 14:48:53.24271619 +0000 UTC m=+481.210599442" Jul 12 14:48:53.517112 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517039 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vqdw2"] Jul 12 14:48:53.517313 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517301 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerName="pull" Jul 12 14:48:53.517352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517314 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerName="pull" Jul 12 14:48:53.517352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517332 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerName="extract" Jul 12 14:48:53.517352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517338 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerName="extract" Jul 12 14:48:53.517352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517349 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerName="util" Jul 12 14:48:53.517468 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517355 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerName="util" Jul 12 14:48:53.517468 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.517399 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf6007b-69f2-4aeb-9b52-6d7136befe0e" containerName="extract" Jul 12 14:48:53.521331 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.521311 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.523103 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.523083 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-946kn\"" Jul 12 14:48:53.529337 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.529315 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vqdw2"] Jul 12 14:48:53.688584 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.688550 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e205c281-8da9-42d4-a514-3c1920b7a64a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vqdw2\" (UID: \"e205c281-8da9-42d4-a514-3c1920b7a64a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.688795 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.688626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf69r\" (UniqueName: \"kubernetes.io/projected/e205c281-8da9-42d4-a514-3c1920b7a64a-kube-api-access-tf69r\") pod \"cert-manager-webhook-6486ffd997-vqdw2\" (UID: \"e205c281-8da9-42d4-a514-3c1920b7a64a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.790222 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.790130 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e205c281-8da9-42d4-a514-3c1920b7a64a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vqdw2\" (UID: \"e205c281-8da9-42d4-a514-3c1920b7a64a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.790222 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.790182 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf69r\" (UniqueName: \"kubernetes.io/projected/e205c281-8da9-42d4-a514-3c1920b7a64a-kube-api-access-tf69r\") pod \"cert-manager-webhook-6486ffd997-vqdw2\" (UID: \"e205c281-8da9-42d4-a514-3c1920b7a64a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.798183 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.798152 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e205c281-8da9-42d4-a514-3c1920b7a64a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vqdw2\" (UID: \"e205c281-8da9-42d4-a514-3c1920b7a64a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.798183 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.798176 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf69r\" (UniqueName: \"kubernetes.io/projected/e205c281-8da9-42d4-a514-3c1920b7a64a-kube-api-access-tf69r\") pod \"cert-manager-webhook-6486ffd997-vqdw2\" (UID: \"e205c281-8da9-42d4-a514-3c1920b7a64a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.830588 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.830551 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:53.955928 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:53.955896 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vqdw2"] Jul 12 14:48:53.958962 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:48:53.958932 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode205c281_8da9_42d4_a514_3c1920b7a64a.slice/crio-c0f21e432f16ba271e511d9e6f5b8dc53c6ca77bff8adaf6b3ed09e7157a808e WatchSource:0}: Error finding container c0f21e432f16ba271e511d9e6f5b8dc53c6ca77bff8adaf6b3ed09e7157a808e: Status 404 returned error can't find the container with id c0f21e432f16ba271e511d9e6f5b8dc53c6ca77bff8adaf6b3ed09e7157a808e Jul 12 14:48:54.215047 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:54.214983 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:54.230578 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:54.230544 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" event={"ID":"e205c281-8da9-42d4-a514-3c1920b7a64a","Type":"ContainerStarted","Data":"ed88db006cf17308427d71a13d981775cef5b6984815e30e616bf495ea610008"} Jul 12 14:48:54.230973 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:54.230595 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:48:54.230973 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:54.230612 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" event={"ID":"e205c281-8da9-42d4-a514-3c1920b7a64a","Type":"ContainerStarted","Data":"c0f21e432f16ba271e511d9e6f5b8dc53c6ca77bff8adaf6b3ed09e7157a808e"} Jul 12 14:48:54.248420 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:54.248367 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" podStartSLOduration=1.248349468 podStartE2EDuration="1.248349468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:48:54.246686828 +0000 UTC m=+482.214570081" watchObservedRunningTime="2026-07-12 14:48:54.248349468 +0000 UTC m=+482.216232720" Jul 12 14:48:54.319914 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:54.317852 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:56.821562 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:56.821530 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:57.467100 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:57.467071 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:57.562719 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:57.562691 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:48:59.675752 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.675720 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2"] Jul 12 14:48:59.678957 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.678940 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:48:59.681066 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.681039 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 14:48:59.681197 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.681045 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zzzvq\"" Jul 12 14:48:59.681611 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.681595 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 14:48:59.686650 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.686627 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2"] Jul 12 14:48:59.732444 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.732408 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx47w\" (UniqueName: \"kubernetes.io/projected/b94d9afd-0f1e-4b1d-b501-6e74981f1090-kube-api-access-gx47w\") pod \"openshift-lws-operator-fd8ccff4c-pfvt2\" (UID: \"b94d9afd-0f1e-4b1d-b501-6e74981f1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:48:59.732632 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.732482 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94d9afd-0f1e-4b1d-b501-6e74981f1090-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pfvt2\" (UID: \"b94d9afd-0f1e-4b1d-b501-6e74981f1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:48:59.833248 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.833099 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx47w\" (UniqueName: \"kubernetes.io/projected/b94d9afd-0f1e-4b1d-b501-6e74981f1090-kube-api-access-gx47w\") pod \"openshift-lws-operator-fd8ccff4c-pfvt2\" (UID: \"b94d9afd-0f1e-4b1d-b501-6e74981f1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:48:59.833420 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.833252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94d9afd-0f1e-4b1d-b501-6e74981f1090-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pfvt2\" (UID: \"b94d9afd-0f1e-4b1d-b501-6e74981f1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:48:59.833621 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.833595 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94d9afd-0f1e-4b1d-b501-6e74981f1090-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pfvt2\" (UID: \"b94d9afd-0f1e-4b1d-b501-6e74981f1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:48:59.840353 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.840333 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx47w\" (UniqueName: \"kubernetes.io/projected/b94d9afd-0f1e-4b1d-b501-6e74981f1090-kube-api-access-gx47w\") pod \"openshift-lws-operator-fd8ccff4c-pfvt2\" (UID: \"b94d9afd-0f1e-4b1d-b501-6e74981f1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:48:59.988288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:48:59.988248 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" Jul 12 14:49:00.109269 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:00.109227 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2"] Jul 12 14:49:00.111432 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:49:00.111408 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb94d9afd_0f1e_4b1d_b501_6e74981f1090.slice/crio-2abf3449e8ccc5c549271d754a487456691eb4644bd85872f82ccbe116916f39 WatchSource:0}: Error finding container 2abf3449e8ccc5c549271d754a487456691eb4644bd85872f82ccbe116916f39: Status 404 returned error can't find the container with id 2abf3449e8ccc5c549271d754a487456691eb4644bd85872f82ccbe116916f39 Jul 12 14:49:00.236756 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:00.236728 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vqdw2" Jul 12 14:49:00.255743 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:00.255712 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" event={"ID":"b94d9afd-0f1e-4b1d-b501-6e74981f1090","Type":"ContainerStarted","Data":"2abf3449e8ccc5c549271d754a487456691eb4644bd85872f82ccbe116916f39"} Jul 12 14:49:02.267713 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:02.267623 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" event={"ID":"b94d9afd-0f1e-4b1d-b501-6e74981f1090","Type":"ContainerStarted","Data":"75d82085db9a15efcefc65a70739c028bd8ac994df171e262a60d91d901ab312"} Jul 12 14:49:02.284674 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:02.284617 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pfvt2" podStartSLOduration=1.462155099 podStartE2EDuration="3.28459738s" totalImagesPullingTime="1.822442281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:49:02.282605421 +0000 UTC m=+490.250488674" watchObservedRunningTime="2026-07-12 14:49:02.28459738 +0000 UTC m=+490.252480635" Jul 12 14:49:02.954098 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:02.954064 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:03.054179 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:03.054148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:04.645880 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.645839 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:04.917923 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.917831 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst"] Jul 12 14:49:04.921566 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.921542 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:04.923459 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.923437 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 14:49:04.923567 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.923482 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 14:49:04.923567 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.923501 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qpjdm\"" Jul 12 14:49:04.928707 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.928665 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst"] Jul 12 14:49:04.978970 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.978941 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:04.979141 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.978976 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:04.979141 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:04.979002 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7cs9\" (UniqueName: \"kubernetes.io/projected/f87e836f-aa0f-46d7-878b-5b224de95149-kube-api-access-s7cs9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.079751 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.079715 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.079751 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.079757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.079968 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.079783 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7cs9\" (UniqueName: \"kubernetes.io/projected/f87e836f-aa0f-46d7-878b-5b224de95149-kube-api-access-s7cs9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.080153 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.080130 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.080195 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.080151 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.087024 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.086993 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7cs9\" (UniqueName: \"kubernetes.io/projected/f87e836f-aa0f-46d7-878b-5b224de95149-kube-api-access-s7cs9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.230883 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.230803 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:05.360491 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:05.360464 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst"] Jul 12 14:49:05.362750 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:49:05.362722 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87e836f_aa0f_46d7_878b_5b224de95149.slice/crio-9ee32a97162277b382d49f8144a757c2a0d27dead1e1d57e80fb6a5bc737fef5 WatchSource:0}: Error finding container 9ee32a97162277b382d49f8144a757c2a0d27dead1e1d57e80fb6a5bc737fef5: Status 404 returned error can't find the container with id 9ee32a97162277b382d49f8144a757c2a0d27dead1e1d57e80fb6a5bc737fef5 Jul 12 14:49:06.288654 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:06.288615 2573 generic.go:356] "Generic (PLEG): container finished" podID="f87e836f-aa0f-46d7-878b-5b224de95149" containerID="e35b5a9b0ce73f7e72cd2be0f666bdb494511783161a86bdcfe0e37023a0234a" exitCode=0 Jul 12 14:49:06.289008 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:06.288701 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" event={"ID":"f87e836f-aa0f-46d7-878b-5b224de95149","Type":"ContainerDied","Data":"e35b5a9b0ce73f7e72cd2be0f666bdb494511783161a86bdcfe0e37023a0234a"} Jul 12 14:49:06.289008 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:06.288748 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" event={"ID":"f87e836f-aa0f-46d7-878b-5b224de95149","Type":"ContainerStarted","Data":"9ee32a97162277b382d49f8144a757c2a0d27dead1e1d57e80fb6a5bc737fef5"} Jul 12 14:49:07.294979 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:07.294946 2573 generic.go:356] "Generic (PLEG): container finished" podID="f87e836f-aa0f-46d7-878b-5b224de95149" containerID="f501c238d2e0bed28593723ec21ff25ca7c73acee83b95652992f6484869d38f" exitCode=0 Jul 12 14:49:07.295393 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:07.295047 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" event={"ID":"f87e836f-aa0f-46d7-878b-5b224de95149","Type":"ContainerDied","Data":"f501c238d2e0bed28593723ec21ff25ca7c73acee83b95652992f6484869d38f"} Jul 12 14:49:08.301571 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:08.301539 2573 generic.go:356] "Generic (PLEG): container finished" podID="f87e836f-aa0f-46d7-878b-5b224de95149" containerID="2fa65eda40f5d233aae2db80a36eabdff5df0b2528626ed1f37998a33a1144f5" exitCode=0 Jul 12 14:49:08.301966 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:08.301616 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" event={"ID":"f87e836f-aa0f-46d7-878b-5b224de95149","Type":"ContainerDied","Data":"2fa65eda40f5d233aae2db80a36eabdff5df0b2528626ed1f37998a33a1144f5"} Jul 12 14:49:09.445784 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.445747 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:09.515461 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.515427 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-bundle\") pod \"f87e836f-aa0f-46d7-878b-5b224de95149\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " Jul 12 14:49:09.515645 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.515509 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-util\") pod \"f87e836f-aa0f-46d7-878b-5b224de95149\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " Jul 12 14:49:09.515645 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.515530 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7cs9\" (UniqueName: \"kubernetes.io/projected/f87e836f-aa0f-46d7-878b-5b224de95149-kube-api-access-s7cs9\") pod \"f87e836f-aa0f-46d7-878b-5b224de95149\" (UID: \"f87e836f-aa0f-46d7-878b-5b224de95149\") " Jul 12 14:49:09.516364 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.516329 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-bundle" (OuterVolumeSpecName: "bundle") pod "f87e836f-aa0f-46d7-878b-5b224de95149" (UID: "f87e836f-aa0f-46d7-878b-5b224de95149"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:49:09.517761 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.517739 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87e836f-aa0f-46d7-878b-5b224de95149-kube-api-access-s7cs9" (OuterVolumeSpecName: "kube-api-access-s7cs9") pod "f87e836f-aa0f-46d7-878b-5b224de95149" (UID: "f87e836f-aa0f-46d7-878b-5b224de95149"). InnerVolumeSpecName "kube-api-access-s7cs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:49:09.520840 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.520820 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-util" (OuterVolumeSpecName: "util") pod "f87e836f-aa0f-46d7-878b-5b224de95149" (UID: "f87e836f-aa0f-46d7-878b-5b224de95149"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:49:09.617003 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.616908 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:49:09.617003 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.616944 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f87e836f-aa0f-46d7-878b-5b224de95149-util\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:49:09.617003 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:09.616953 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7cs9\" (UniqueName: \"kubernetes.io/projected/f87e836f-aa0f-46d7-878b-5b224de95149-kube-api-access-s7cs9\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:49:10.312135 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:10.312105 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" Jul 12 14:49:10.312339 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:10.312100 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcpzst" event={"ID":"f87e836f-aa0f-46d7-878b-5b224de95149","Type":"ContainerDied","Data":"9ee32a97162277b382d49f8144a757c2a0d27dead1e1d57e80fb6a5bc737fef5"} Jul 12 14:49:10.312339 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:10.312233 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ee32a97162277b382d49f8144a757c2a0d27dead1e1d57e80fb6a5bc737fef5" Jul 12 14:49:17.686856 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:17.686812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:17.776571 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:17.776532 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:18.282707 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.282674 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:18.472680 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.472647 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:18.589977 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.589878 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:18.642213 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642172 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv"] Jul 12 14:49:18.642528 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642510 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87e836f-aa0f-46d7-878b-5b224de95149" containerName="util" Jul 12 14:49:18.642610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642531 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87e836f-aa0f-46d7-878b-5b224de95149" containerName="util" Jul 12 14:49:18.642610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642542 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87e836f-aa0f-46d7-878b-5b224de95149" containerName="extract" Jul 12 14:49:18.642610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642549 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87e836f-aa0f-46d7-878b-5b224de95149" containerName="extract" Jul 12 14:49:18.642610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642566 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87e836f-aa0f-46d7-878b-5b224de95149" containerName="pull" Jul 12 14:49:18.642610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642576 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87e836f-aa0f-46d7-878b-5b224de95149" containerName="pull" Jul 12 14:49:18.642843 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.642665 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87e836f-aa0f-46d7-878b-5b224de95149" containerName="extract" Jul 12 14:49:18.646758 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.646735 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.648892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.648870 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 14:49:18.648892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.648884 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 14:49:18.649113 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.648983 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 14:49:18.649180 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.649144 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 14:49:18.649340 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.649325 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ffwdm\"" Jul 12 14:49:18.660201 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.660180 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv"] Jul 12 14:49:18.691945 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.691913 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bb28d76-211c-44e3-9651-07b03bbde796-tmp\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.692395 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.691967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bb28d76-211c-44e3-9651-07b03bbde796-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.692395 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.692023 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkmcg\" (UniqueName: \"kubernetes.io/projected/9bb28d76-211c-44e3-9651-07b03bbde796-kube-api-access-dkmcg\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.692395 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.692098 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bb28d76-211c-44e3-9651-07b03bbde796-manifests\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.692395 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.692143 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bb28d76-211c-44e3-9651-07b03bbde796-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.781316 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.781281 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:18.793398 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.793368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bb28d76-211c-44e3-9651-07b03bbde796-manifests\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.793530 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.793413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bb28d76-211c-44e3-9651-07b03bbde796-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.793530 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.793453 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bb28d76-211c-44e3-9651-07b03bbde796-tmp\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.793530 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.793477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bb28d76-211c-44e3-9651-07b03bbde796-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.793530 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.793498 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkmcg\" (UniqueName: \"kubernetes.io/projected/9bb28d76-211c-44e3-9651-07b03bbde796-kube-api-access-dkmcg\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.793838 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.793813 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bb28d76-211c-44e3-9651-07b03bbde796-manifests\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.795695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.795673 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bb28d76-211c-44e3-9651-07b03bbde796-tmp\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.795780 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.795725 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bb28d76-211c-44e3-9651-07b03bbde796-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.795895 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.795878 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bb28d76-211c-44e3-9651-07b03bbde796-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.802390 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.802364 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkmcg\" (UniqueName: \"kubernetes.io/projected/9bb28d76-211c-44e3-9651-07b03bbde796-kube-api-access-dkmcg\") pod \"opendatahub-operator-controller-manager-6d4d9f5575-88csv\" (UID: \"9bb28d76-211c-44e3-9651-07b03bbde796\") " pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.885850 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.885822 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:18.957048 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.956994 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:18.981461 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:18.981431 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:19.080965 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:19.080933 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:19.100264 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:19.100238 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv"] Jul 12 14:49:19.103587 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:49:19.103564 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb28d76_211c_44e3_9651_07b03bbde796.slice/crio-83ec9d2f36b0fac940bf4e09bfc544a396f618fae7a8ba49a3d8e0583d0aeefd WatchSource:0}: Error finding container 83ec9d2f36b0fac940bf4e09bfc544a396f618fae7a8ba49a3d8e0583d0aeefd: Status 404 returned error can't find the container with id 83ec9d2f36b0fac940bf4e09bfc544a396f618fae7a8ba49a3d8e0583d0aeefd Jul 12 14:49:19.186263 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:19.186156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:19.284717 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:19.284682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:19.350597 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:19.350563 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" event={"ID":"9bb28d76-211c-44e3-9651-07b03bbde796","Type":"ContainerStarted","Data":"83ec9d2f36b0fac940bf4e09bfc544a396f618fae7a8ba49a3d8e0583d0aeefd"} Jul 12 14:49:19.395395 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:19.395354 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:22.367161 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:22.367076 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" event={"ID":"9bb28d76-211c-44e3-9651-07b03bbde796","Type":"ContainerStarted","Data":"611edd362afe6b02bb21e3ba99f3926a659d542eeefa1a6bd78cc2c1a07efda8"} Jul 12 14:49:22.903404 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:22.903374 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:23.022469 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:23.022439 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:23.377779 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:23.377748 2573 generic.go:356] "Generic (PLEG): container finished" podID="9bb28d76-211c-44e3-9651-07b03bbde796" containerID="611edd362afe6b02bb21e3ba99f3926a659d542eeefa1a6bd78cc2c1a07efda8" exitCode=0 Jul 12 14:49:23.378297 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:23.377802 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" event={"ID":"9bb28d76-211c-44e3-9651-07b03bbde796","Type":"ContainerDied","Data":"611edd362afe6b02bb21e3ba99f3926a659d542eeefa1a6bd78cc2c1a07efda8"} Jul 12 14:49:24.384863 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:24.384827 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" event={"ID":"9bb28d76-211c-44e3-9651-07b03bbde796","Type":"ContainerStarted","Data":"9bd2159fe405c15d1fe279ae75acd65d17cb3eb72d4c81e3211867d8854477fc"} Jul 12 14:49:24.385360 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:24.384948 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:24.404320 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:24.404276 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" podStartSLOduration=2.220803728 podStartE2EDuration="6.404260075s" totalImagesPullingTime="3.083284048s" totalInitContainerRuntime="1.100172299s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:49:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:49:24.403421295 +0000 UTC m=+512.371304548" watchObservedRunningTime="2026-07-12 14:49:24.404260075 +0000 UTC m=+512.372143326" Jul 12 14:49:28.319988 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:28.319949 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:28.419828 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:28.419790 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:29.798541 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:29.798501 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:29.917982 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:29.917947 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:35.391413 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:35.391377 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d4d9f5575-88csv" Jul 12 14:49:37.563621 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:37.563584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:37.687034 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:37.686986 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:37.909194 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:37.909165 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:38.083094 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.083061 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:38.253927 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.253839 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf"] Jul 12 14:49:38.264430 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.264402 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.268403 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.268375 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 14:49:38.269002 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.268977 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 14:49:38.269142 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.268974 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 12 14:49:38.269142 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.269104 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l5qxl\"" Jul 12 14:49:38.269142 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.269047 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 12 14:49:38.284105 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.284072 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf"] Jul 12 14:49:38.372778 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.372740 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrq8d\" (UniqueName: \"kubernetes.io/projected/5075b719-f93f-4559-beda-a6c11a335af9-kube-api-access-rrq8d\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.373073 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.372793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5075b719-f93f-4559-beda-a6c11a335af9-tmp\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.373073 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.372823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5075b719-f93f-4559-beda-a6c11a335af9-tls-certs\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.473510 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.473473 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrq8d\" (UniqueName: \"kubernetes.io/projected/5075b719-f93f-4559-beda-a6c11a335af9-kube-api-access-rrq8d\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.473683 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.473528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5075b719-f93f-4559-beda-a6c11a335af9-tmp\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.473683 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.473660 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5075b719-f93f-4559-beda-a6c11a335af9-tls-certs\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.475909 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.475883 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5075b719-f93f-4559-beda-a6c11a335af9-tmp\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.476234 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.476213 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5075b719-f93f-4559-beda-a6c11a335af9-tls-certs\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.481977 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.481956 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrq8d\" (UniqueName: \"kubernetes.io/projected/5075b719-f93f-4559-beda-a6c11a335af9-kube-api-access-rrq8d\") pod \"kube-auth-proxy-6b855bfc69-bdqdf\" (UID: \"5075b719-f93f-4559-beda-a6c11a335af9\") " pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.574137 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.574043 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" Jul 12 14:49:38.583071 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.583031 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:38.696048 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.695875 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:38.753364 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.753336 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf"] Jul 12 14:49:38.755296 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:49:38.755267 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5075b719_f93f_4559_beda_a6c11a335af9.slice/crio-76f9aa9268805a01647235107546fe01c59a2c4e61bf43d4ad9474bc690a0760 WatchSource:0}: Error finding container 76f9aa9268805a01647235107546fe01c59a2c4e61bf43d4ad9474bc690a0760: Status 404 returned error can't find the container with id 76f9aa9268805a01647235107546fe01c59a2c4e61bf43d4ad9474bc690a0760 Jul 12 14:49:38.784938 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.784897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:38.934109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:38.934078 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:39.454728 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.454688 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" event={"ID":"5075b719-f93f-4559-beda-a6c11a335af9","Type":"ContainerStarted","Data":"76f9aa9268805a01647235107546fe01c59a2c4e61bf43d4ad9474bc690a0760"} Jul 12 14:49:39.465877 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.465844 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-56fs9"] Jul 12 14:49:39.469589 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.469526 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:39.471950 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.471927 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 12 14:49:39.485074 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.485005 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:39.485723 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.485152 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-56fs9"] Jul 12 14:49:39.584578 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.584407 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckg2z\" (UniqueName: \"kubernetes.io/projected/65758d6f-63fb-4b2a-b915-6fa74894d187-kube-api-access-ckg2z\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:39.584578 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.584520 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:39.659714 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.659678 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:39.686569 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.685278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckg2z\" (UniqueName: \"kubernetes.io/projected/65758d6f-63fb-4b2a-b915-6fa74894d187-kube-api-access-ckg2z\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:39.686569 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.685388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:39.686569 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:49:39.685533 2573 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 12 14:49:39.686569 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:49:39.685603 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs podName:65758d6f-63fb-4b2a-b915-6fa74894d187 nodeName:}" failed. No retries permitted until 2026-07-12 14:49:40.185581392 +0000 UTC m=+528.153464623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs") pod "model-serving-api-54d89bd5fc-56fs9" (UID: "65758d6f-63fb-4b2a-b915-6fa74894d187") : secret "model-serving-api-tls" not found Jul 12 14:49:39.699369 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.699318 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckg2z\" (UniqueName: \"kubernetes.io/projected/65758d6f-63fb-4b2a-b915-6fa74894d187-kube-api-access-ckg2z\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:39.764489 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:39.764384 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:40.190265 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:40.190225 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:40.190469 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:49:40.190446 2573 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 12 14:49:40.190551 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:49:40.190540 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs podName:65758d6f-63fb-4b2a-b915-6fa74894d187 nodeName:}" failed. No retries permitted until 2026-07-12 14:49:41.190520587 +0000 UTC m=+529.158403822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs") pod "model-serving-api-54d89bd5fc-56fs9" (UID: "65758d6f-63fb-4b2a-b915-6fa74894d187") : secret "model-serving-api-tls" not found Jul 12 14:49:40.422303 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:40.422270 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:40.724286 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:40.724245 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:41.201325 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:41.201298 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:41.203702 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:41.203672 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65758d6f-63fb-4b2a-b915-6fa74894d187-tls-certs\") pod \"model-serving-api-54d89bd5fc-56fs9\" (UID: \"65758d6f-63fb-4b2a-b915-6fa74894d187\") " pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:41.285118 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:41.285082 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:41.422269 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:41.422242 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-56fs9"] Jul 12 14:49:41.424334 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:49:41.424307 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65758d6f_63fb_4b2a_b915_6fa74894d187.slice/crio-71b72b81fd18d0294404c41e23b18dffbcf5e90793a690b7e1e3fe7029ea55b6 WatchSource:0}: Error finding container 71b72b81fd18d0294404c41e23b18dffbcf5e90793a690b7e1e3fe7029ea55b6: Status 404 returned error can't find the container with id 71b72b81fd18d0294404c41e23b18dffbcf5e90793a690b7e1e3fe7029ea55b6 Jul 12 14:49:41.472837 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:41.472795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" event={"ID":"65758d6f-63fb-4b2a-b915-6fa74894d187","Type":"ContainerStarted","Data":"71b72b81fd18d0294404c41e23b18dffbcf5e90793a690b7e1e3fe7029ea55b6"} Jul 12 14:49:41.474426 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:41.474392 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" event={"ID":"5075b719-f93f-4559-beda-a6c11a335af9","Type":"ContainerStarted","Data":"219f4dcc8f92e014f7c972a82b0f538e6c4cfe60725cc00bd566dabc58a97d67"} Jul 12 14:49:41.489332 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:41.489276 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b855bfc69-bdqdf" podStartSLOduration=1.11416305 podStartE2EDuration="3.489259445s" totalImagesPullingTime="2.375096395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:49:41.488518586 +0000 UTC m=+529.456401847" watchObservedRunningTime="2026-07-12 14:49:41.489259445 +0000 UTC m=+529.457142697" Jul 12 14:49:43.487555 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:43.487521 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" event={"ID":"65758d6f-63fb-4b2a-b915-6fa74894d187","Type":"ContainerStarted","Data":"5599c473985b29fd27d92ebdaf2e9898db438ebac07ec7b18c04f6dbf0db8f1b"} Jul 12 14:49:43.487941 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:43.487616 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:43.503022 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:43.502970 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" podStartSLOduration=3.047858212 podStartE2EDuration="4.502952088s" totalImagesPullingTime="1.455093876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:49:43.502779857 +0000 UTC m=+531.470663111" watchObservedRunningTime="2026-07-12 14:49:43.502952088 +0000 UTC m=+531.470835341" Jul 12 14:49:44.459221 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:44.459181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:44.558184 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:44.558152 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:45.056118 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.056085 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt"] Jul 12 14:49:45.059487 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.059468 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.062138 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.062114 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 14:49:45.062242 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.062114 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-l4mrp\"" Jul 12 14:49:45.072632 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.072605 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt"] Jul 12 14:49:45.093682 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.093647 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:45.235321 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235281 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235514 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235346 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235514 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235514 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235395 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f9fl\" (UniqueName: \"kubernetes.io/projected/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-kube-api-access-8f9fl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235514 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235485 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235665 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235665 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235574 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235665 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235639 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.235665 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.235660 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.294052 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.293992 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:45.336580 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.336489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.336814 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.336791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f9fl\" (UniqueName: \"kubernetes.io/projected/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-kube-api-access-8f9fl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.336948 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.336833 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.337228 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337210 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.337362 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.337495 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337463 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.337609 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.337710 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.337825 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337810 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.337949 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.338240 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.338214 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.338503 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.338486 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.338757 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.337943 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.339224 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.339199 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.351792 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.351755 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f9fl\" (UniqueName: \"kubernetes.io/projected/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-kube-api-access-8f9fl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.352378 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.352351 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.353460 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.353430 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.353847 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.353824 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee867255-1dd5-465b-81ab-e7f2d4b5d5cb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t6vpt\" (UID: \"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.370085 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.370051 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:45.417062 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.417024 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:45.542787 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:45.542759 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt"] Jul 12 14:49:45.543787 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:49:45.543759 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee867255_1dd5_465b_81ab_e7f2d4b5d5cb.slice/crio-7152875756a362c2974cc207f641dbcc681c60c93c8d1215d57145ecc56bce5c WatchSource:0}: Error finding container 7152875756a362c2974cc207f641dbcc681c60c93c8d1215d57145ecc56bce5c: Status 404 returned error can't find the container with id 7152875756a362c2974cc207f641dbcc681c60c93c8d1215d57145ecc56bce5c Jul 12 14:49:46.510111 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.510061 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" event={"ID":"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb","Type":"ContainerStarted","Data":"7152875756a362c2974cc207f641dbcc681c60c93c8d1215d57145ecc56bce5c"} Jul 12 14:49:46.736214 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.736166 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd"] Jul 12 14:49:46.739799 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.739777 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.742751 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.742725 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-m6zmm\"" Jul 12 14:49:46.743225 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.742909 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 14:49:46.743225 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.742909 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 14:49:46.743225 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.743032 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 14:49:46.753186 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.752482 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd"] Jul 12 14:49:46.854888 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.854790 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af94111a-b8e8-4543-b6b2-b65f73191f24-cert\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.854888 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.854843 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/af94111a-b8e8-4543-b6b2-b65f73191f24-metrics-cert\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.854888 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.854868 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9twqt\" (UniqueName: \"kubernetes.io/projected/af94111a-b8e8-4543-b6b2-b65f73191f24-kube-api-access-9twqt\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.855217 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.855029 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/af94111a-b8e8-4543-b6b2-b65f73191f24-manager-config\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.956028 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.955971 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af94111a-b8e8-4543-b6b2-b65f73191f24-cert\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.956028 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.956009 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/af94111a-b8e8-4543-b6b2-b65f73191f24-metrics-cert\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.956250 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.956058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9twqt\" (UniqueName: \"kubernetes.io/projected/af94111a-b8e8-4543-b6b2-b65f73191f24-kube-api-access-9twqt\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.956250 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.956218 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/af94111a-b8e8-4543-b6b2-b65f73191f24-manager-config\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.956783 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.956758 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/af94111a-b8e8-4543-b6b2-b65f73191f24-manager-config\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.958463 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.958435 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af94111a-b8e8-4543-b6b2-b65f73191f24-cert\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.958463 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.958450 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/af94111a-b8e8-4543-b6b2-b65f73191f24-metrics-cert\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:46.964872 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:46.964846 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9twqt\" (UniqueName: \"kubernetes.io/projected/af94111a-b8e8-4543-b6b2-b65f73191f24-kube-api-access-9twqt\") pod \"lws-controller-manager-5f9f5d4d9d-2mszd\" (UID: \"af94111a-b8e8-4543-b6b2-b65f73191f24\") " pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:47.054680 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:47.054637 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:47.207843 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:47.207819 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd"] Jul 12 14:49:47.210003 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:49:47.209966 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf94111a_b8e8_4543_b6b2_b65f73191f24.slice/crio-df9e6ae629d5fa5ca3fe5aaaac4643697759f23946221c832a9923d664770ebb WatchSource:0}: Error finding container df9e6ae629d5fa5ca3fe5aaaac4643697759f23946221c832a9923d664770ebb: Status 404 returned error can't find the container with id df9e6ae629d5fa5ca3fe5aaaac4643697759f23946221c832a9923d664770ebb Jul 12 14:49:47.517232 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:47.517196 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" event={"ID":"af94111a-b8e8-4543-b6b2-b65f73191f24","Type":"ContainerStarted","Data":"df9e6ae629d5fa5ca3fe5aaaac4643697759f23946221c832a9923d664770ebb"} Jul 12 14:49:48.777401 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:48.777362 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 14:49:48.777649 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:48.777444 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 14:49:48.777649 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:48.777477 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 14:49:49.080297 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:49.080147 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:49.198239 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:49.198182 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:49.523513 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:49.523432 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:49.530212 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:49.530177 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" event={"ID":"ee867255-1dd5-465b-81ab-e7f2d4b5d5cb","Type":"ContainerStarted","Data":"958b665354d6adc79c288eba32688afb99573a22fe15fb72e1c87680c93b026b"} Jul 12 14:49:49.552747 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:49.552690 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" podStartSLOduration=1.321362498 podStartE2EDuration="4.552668116s" totalImagesPullingTime="3.231305618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:49:49.551383139 +0000 UTC m=+537.519266428" watchObservedRunningTime="2026-07-12 14:49:49.552668116 +0000 UTC m=+537.520551370" Jul 12 14:49:49.622947 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:49.622897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:49.726073 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:49.725996 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:50.340710 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.340677 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:50.370653 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.370610 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:50.376703 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.376670 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:50.405616 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.405561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:50.466089 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.466053 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:50.540686 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.540645 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" event={"ID":"af94111a-b8e8-4543-b6b2-b65f73191f24","Type":"ContainerStarted","Data":"ece8f351fff4108105778f1987a0402f509493b0441f3f6a9b372aee71bdafb6"} Jul 12 14:49:50.540899 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.540710 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:50.540899 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.540773 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:49:50.541763 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.541737 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t6vpt" Jul 12 14:49:50.571993 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:50.571939 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" podStartSLOduration=1.9362472990000001 podStartE2EDuration="4.571922863s" totalImagesPullingTime="2.635675564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:49:50.56999634 +0000 UTC m=+538.537879592" watchObservedRunningTime="2026-07-12 14:49:50.571922863 +0000 UTC m=+538.539806116" Jul 12 14:49:53.966298 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:53.966258 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:54.027153 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:54.027116 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:54.497638 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:54.497606 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-56fs9" Jul 12 14:49:55.068784 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:55.068748 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:55.124973 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:55.124937 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:57.226149 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:57.226113 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:49:57.322001 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:49:57.321968 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:01.546117 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:01.546084 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f9f5d4d9d-2mszd" Jul 12 14:50:11.788621 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:11.788582 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:11.889389 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:11.889355 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:27.228746 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:27.228708 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:32.543405 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:32.543368 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:32.600033 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:32.599977 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:38.987964 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:38.987920 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:39.125539 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:39.125503 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:39.226241 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:39.226208 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:52.489177 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:52.489147 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:50:52.490190 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:52.490167 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:50:52.495637 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:52.495618 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:50:52.496422 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:52.496403 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:50:54.286267 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.286231 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7"] Jul 12 14:50:54.289652 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.289634 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.291745 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.291718 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 14:50:54.291886 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.291746 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fcgxw\"" Jul 12 14:50:54.291886 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.291718 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 14:50:54.296937 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.296915 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7"] Jul 12 14:50:54.357772 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.357743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.357931 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.357807 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6n94\" (UniqueName: \"kubernetes.io/projected/8507da11-9656-4b5f-9f36-105cb2fdae9a-kube-api-access-d6n94\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.357931 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.357912 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.459249 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.459205 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.459433 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.459276 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.459433 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.459327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6n94\" (UniqueName: \"kubernetes.io/projected/8507da11-9656-4b5f-9f36-105cb2fdae9a-kube-api-access-d6n94\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.459749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.459724 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.459749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.459739 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.469467 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.469441 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6n94\" (UniqueName: \"kubernetes.io/projected/8507da11-9656-4b5f-9f36-105cb2fdae9a-kube-api-access-d6n94\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.599836 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.599757 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:54.681489 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.681378 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb"] Jul 12 14:50:54.687519 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.687496 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.692493 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.692456 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb"] Jul 12 14:50:54.731724 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.731700 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7"] Jul 12 14:50:54.734062 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:50:54.734035 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8507da11_9656_4b5f_9f36_105cb2fdae9a.slice/crio-4d3725e919651d69434426376ec17625d0b9ec5e01ebfd3e353dea8a51e0f186 WatchSource:0}: Error finding container 4d3725e919651d69434426376ec17625d0b9ec5e01ebfd3e353dea8a51e0f186: Status 404 returned error can't find the container with id 4d3725e919651d69434426376ec17625d0b9ec5e01ebfd3e353dea8a51e0f186 Jul 12 14:50:54.761610 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.761588 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.761725 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.761650 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx49p\" (UniqueName: \"kubernetes.io/projected/309ebb07-a049-4716-a791-e88f343d62d9-kube-api-access-nx49p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.761725 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.761711 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.838213 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.838182 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" event={"ID":"8507da11-9656-4b5f-9f36-105cb2fdae9a","Type":"ContainerStarted","Data":"9897ffceea6f43cf4ff4940cca118c5f8c2b17ba1cd704428fdfec3e6969bbfc"} Jul 12 14:50:54.838307 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.838238 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" event={"ID":"8507da11-9656-4b5f-9f36-105cb2fdae9a","Type":"ContainerStarted","Data":"4d3725e919651d69434426376ec17625d0b9ec5e01ebfd3e353dea8a51e0f186"} Jul 12 14:50:54.862853 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.862794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.862954 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.862857 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.862954 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.862915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx49p\" (UniqueName: \"kubernetes.io/projected/309ebb07-a049-4716-a791-e88f343d62d9-kube-api-access-nx49p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.863200 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.863178 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.863243 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.863230 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.869840 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.869814 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx49p\" (UniqueName: \"kubernetes.io/projected/309ebb07-a049-4716-a791-e88f343d62d9-kube-api-access-nx49p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:54.999523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:54.999481 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:55.124749 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.124725 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb"] Jul 12 14:50:55.126480 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:50:55.126452 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod309ebb07_a049_4716_a791_e88f343d62d9.slice/crio-ba3579bc831caddfeb46d4a16f9bac2b7b43317c8f0dd04a0c48f5007398f8bf WatchSource:0}: Error finding container ba3579bc831caddfeb46d4a16f9bac2b7b43317c8f0dd04a0c48f5007398f8bf: Status 404 returned error can't find the container with id ba3579bc831caddfeb46d4a16f9bac2b7b43317c8f0dd04a0c48f5007398f8bf Jul 12 14:50:55.278338 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.278296 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49"] Jul 12 14:50:55.281843 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.281824 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.292699 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.292678 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49"] Jul 12 14:50:55.366439 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.366392 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.366616 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.366477 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.366616 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.366498 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh2dr\" (UniqueName: \"kubernetes.io/projected/f48a30dc-8e56-4d30-a219-75661726d3c5-kube-api-access-bh2dr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.467446 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.467399 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.467446 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.467450 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh2dr\" (UniqueName: \"kubernetes.io/projected/f48a30dc-8e56-4d30-a219-75661726d3c5-kube-api-access-bh2dr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.467724 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.467527 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.467840 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.467816 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.467902 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.467884 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.474565 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.474539 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh2dr\" (UniqueName: \"kubernetes.io/projected/f48a30dc-8e56-4d30-a219-75661726d3c5-kube-api-access-bh2dr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.594802 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.594763 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:55.722679 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.722647 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49"] Jul 12 14:50:55.725048 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:50:55.724995 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf48a30dc_8e56_4d30_a219_75661726d3c5.slice/crio-e712813cbdd7d8aba747ee0d7c2bc9cb43545970fd02fa6cc0c9f7c7100bbdd5 WatchSource:0}: Error finding container e712813cbdd7d8aba747ee0d7c2bc9cb43545970fd02fa6cc0c9f7c7100bbdd5: Status 404 returned error can't find the container with id e712813cbdd7d8aba747ee0d7c2bc9cb43545970fd02fa6cc0c9f7c7100bbdd5 Jul 12 14:50:55.844795 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.844763 2573 generic.go:356] "Generic (PLEG): container finished" podID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerID="b08bf6d227aafd65e469a0ef515347481083609d0020efafaba08d6e717db450" exitCode=0 Jul 12 14:50:55.845029 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.844842 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" event={"ID":"f48a30dc-8e56-4d30-a219-75661726d3c5","Type":"ContainerDied","Data":"b08bf6d227aafd65e469a0ef515347481083609d0020efafaba08d6e717db450"} Jul 12 14:50:55.845029 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.844892 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" event={"ID":"f48a30dc-8e56-4d30-a219-75661726d3c5","Type":"ContainerStarted","Data":"e712813cbdd7d8aba747ee0d7c2bc9cb43545970fd02fa6cc0c9f7c7100bbdd5"} Jul 12 14:50:55.846367 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.846343 2573 generic.go:356] "Generic (PLEG): container finished" podID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerID="9897ffceea6f43cf4ff4940cca118c5f8c2b17ba1cd704428fdfec3e6969bbfc" exitCode=0 Jul 12 14:50:55.846441 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.846419 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" event={"ID":"8507da11-9656-4b5f-9f36-105cb2fdae9a","Type":"ContainerDied","Data":"9897ffceea6f43cf4ff4940cca118c5f8c2b17ba1cd704428fdfec3e6969bbfc"} Jul 12 14:50:55.847983 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.847962 2573 generic.go:356] "Generic (PLEG): container finished" podID="309ebb07-a049-4716-a791-e88f343d62d9" containerID="aa4a607d6aca767e67c905dfb37ba73723af1163a920c3ac2e9763bd08ae6762" exitCode=0 Jul 12 14:50:55.848090 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.847993 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" event={"ID":"309ebb07-a049-4716-a791-e88f343d62d9","Type":"ContainerDied","Data":"aa4a607d6aca767e67c905dfb37ba73723af1163a920c3ac2e9763bd08ae6762"} Jul 12 14:50:55.848090 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.848049 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" event={"ID":"309ebb07-a049-4716-a791-e88f343d62d9","Type":"ContainerStarted","Data":"ba3579bc831caddfeb46d4a16f9bac2b7b43317c8f0dd04a0c48f5007398f8bf"} Jul 12 14:50:55.880627 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.880594 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7"] Jul 12 14:50:55.884116 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.884095 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:55.892339 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.892302 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7"] Jul 12 14:50:55.971208 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.971167 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:55.971409 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.971237 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-478cc\" (UniqueName: \"kubernetes.io/projected/cf2e9876-c6ef-4356-a884-14ec05b98a45-kube-api-access-478cc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:55.971409 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:55.971267 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.072727 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.072641 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.072727 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.072723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.073109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.072757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-478cc\" (UniqueName: \"kubernetes.io/projected/cf2e9876-c6ef-4356-a884-14ec05b98a45-kube-api-access-478cc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.073109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.073050 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.073109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.073065 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.081544 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.081521 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-478cc\" (UniqueName: \"kubernetes.io/projected/cf2e9876-c6ef-4356-a884-14ec05b98a45-kube-api-access-478cc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.194408 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.194371 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:50:56.325164 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.325139 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7"] Jul 12 14:50:56.327158 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:50:56.327130 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2e9876_c6ef_4356_a884_14ec05b98a45.slice/crio-73cbe62fa7a56f301b2b8975d2ce31f7807889c59b7b679798e1893a598c3e8f WatchSource:0}: Error finding container 73cbe62fa7a56f301b2b8975d2ce31f7807889c59b7b679798e1893a598c3e8f: Status 404 returned error can't find the container with id 73cbe62fa7a56f301b2b8975d2ce31f7807889c59b7b679798e1893a598c3e8f Jul 12 14:50:56.854559 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.854527 2573 generic.go:356] "Generic (PLEG): container finished" podID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerID="9a5456f518964dca5f1bc1246019fd6f60c5e3235dc80985c7d7edb3ff0db676" exitCode=0 Jul 12 14:50:56.854755 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.854617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" event={"ID":"f48a30dc-8e56-4d30-a219-75661726d3c5","Type":"ContainerDied","Data":"9a5456f518964dca5f1bc1246019fd6f60c5e3235dc80985c7d7edb3ff0db676"} Jul 12 14:50:56.856458 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.856434 2573 generic.go:356] "Generic (PLEG): container finished" podID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerID="1087192dd4168e4ac0b982bcf1da97503c76d8b95879efb997b94064ff68e5b3" exitCode=0 Jul 12 14:50:56.856554 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.856477 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" event={"ID":"8507da11-9656-4b5f-9f36-105cb2fdae9a","Type":"ContainerDied","Data":"1087192dd4168e4ac0b982bcf1da97503c76d8b95879efb997b94064ff68e5b3"} Jul 12 14:50:56.858131 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.858114 2573 generic.go:356] "Generic (PLEG): container finished" podID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerID="0bab81d8e026daf58a6b8863a3ad2cbb6d8b0b3879c6749879b7df66d60ad9f6" exitCode=0 Jul 12 14:50:56.858215 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.858196 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" event={"ID":"cf2e9876-c6ef-4356-a884-14ec05b98a45","Type":"ContainerDied","Data":"0bab81d8e026daf58a6b8863a3ad2cbb6d8b0b3879c6749879b7df66d60ad9f6"} Jul 12 14:50:56.858267 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.858239 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" event={"ID":"cf2e9876-c6ef-4356-a884-14ec05b98a45","Type":"ContainerStarted","Data":"73cbe62fa7a56f301b2b8975d2ce31f7807889c59b7b679798e1893a598c3e8f"} Jul 12 14:50:56.860154 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.860135 2573 generic.go:356] "Generic (PLEG): container finished" podID="309ebb07-a049-4716-a791-e88f343d62d9" containerID="0840eff340635f6d7b93a46aebcef9b4ac08d810b6b1642fad44c98b4ae97ddc" exitCode=0 Jul 12 14:50:56.860244 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:56.860163 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" event={"ID":"309ebb07-a049-4716-a791-e88f343d62d9","Type":"ContainerDied","Data":"0840eff340635f6d7b93a46aebcef9b4ac08d810b6b1642fad44c98b4ae97ddc"} Jul 12 14:50:57.867882 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:57.867802 2573 generic.go:356] "Generic (PLEG): container finished" podID="309ebb07-a049-4716-a791-e88f343d62d9" containerID="0a90e6734cc76110fbfb6cb1cbf65b74de219e36c02a3046761044030530d4e1" exitCode=0 Jul 12 14:50:57.868301 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:57.867885 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" event={"ID":"309ebb07-a049-4716-a791-e88f343d62d9","Type":"ContainerDied","Data":"0a90e6734cc76110fbfb6cb1cbf65b74de219e36c02a3046761044030530d4e1"} Jul 12 14:50:57.870044 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:57.870008 2573 generic.go:356] "Generic (PLEG): container finished" podID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerID="c6ef5f0c169184029a032aae614794f2c7a144b75463874926eeaf53879667ba" exitCode=0 Jul 12 14:50:57.870153 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:57.870100 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" event={"ID":"f48a30dc-8e56-4d30-a219-75661726d3c5","Type":"ContainerDied","Data":"c6ef5f0c169184029a032aae614794f2c7a144b75463874926eeaf53879667ba"} Jul 12 14:50:57.872147 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:57.872128 2573 generic.go:356] "Generic (PLEG): container finished" podID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerID="63ac2d1aaa027ff9415ec711beac0d07df2b0f028243d7e193a52580b972f473" exitCode=0 Jul 12 14:50:57.872251 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:57.872158 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" event={"ID":"8507da11-9656-4b5f-9f36-105cb2fdae9a","Type":"ContainerDied","Data":"63ac2d1aaa027ff9415ec711beac0d07df2b0f028243d7e193a52580b972f473"} Jul 12 14:50:58.742832 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:58.742802 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:58.878183 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:58.878147 2573 generic.go:356] "Generic (PLEG): container finished" podID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerID="e409a621859c2466bb9772db52a21c605f5d88301c5a36b0af650ee9a7715a8d" exitCode=0 Jul 12 14:50:58.878594 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:58.878188 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" event={"ID":"cf2e9876-c6ef-4356-a884-14ec05b98a45","Type":"ContainerDied","Data":"e409a621859c2466bb9772db52a21c605f5d88301c5a36b0af650ee9a7715a8d"} Jul 12 14:50:59.019898 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.019870 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:50:59.101217 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.101192 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh2dr\" (UniqueName: \"kubernetes.io/projected/f48a30dc-8e56-4d30-a219-75661726d3c5-kube-api-access-bh2dr\") pod \"f48a30dc-8e56-4d30-a219-75661726d3c5\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " Jul 12 14:50:59.101335 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.101230 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-bundle\") pod \"f48a30dc-8e56-4d30-a219-75661726d3c5\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " Jul 12 14:50:59.101335 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.101306 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-util\") pod \"f48a30dc-8e56-4d30-a219-75661726d3c5\" (UID: \"f48a30dc-8e56-4d30-a219-75661726d3c5\") " Jul 12 14:50:59.101819 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.101795 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-bundle" (OuterVolumeSpecName: "bundle") pod "f48a30dc-8e56-4d30-a219-75661726d3c5" (UID: "f48a30dc-8e56-4d30-a219-75661726d3c5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:50:59.103394 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.103365 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f48a30dc-8e56-4d30-a219-75661726d3c5-kube-api-access-bh2dr" (OuterVolumeSpecName: "kube-api-access-bh2dr") pod "f48a30dc-8e56-4d30-a219-75661726d3c5" (UID: "f48a30dc-8e56-4d30-a219-75661726d3c5"). InnerVolumeSpecName "kube-api-access-bh2dr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:50:59.105958 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.105934 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-util" (OuterVolumeSpecName: "util") pod "f48a30dc-8e56-4d30-a219-75661726d3c5" (UID: "f48a30dc-8e56-4d30-a219-75661726d3c5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:50:59.131231 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.131208 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:59.133877 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.133859 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:59.202321 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202281 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-util\") pod \"8507da11-9656-4b5f-9f36-105cb2fdae9a\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " Jul 12 14:50:59.202515 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202340 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-util\") pod \"309ebb07-a049-4716-a791-e88f343d62d9\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " Jul 12 14:50:59.202515 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202368 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6n94\" (UniqueName: \"kubernetes.io/projected/8507da11-9656-4b5f-9f36-105cb2fdae9a-kube-api-access-d6n94\") pod \"8507da11-9656-4b5f-9f36-105cb2fdae9a\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " Jul 12 14:50:59.202515 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202389 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx49p\" (UniqueName: \"kubernetes.io/projected/309ebb07-a049-4716-a791-e88f343d62d9-kube-api-access-nx49p\") pod \"309ebb07-a049-4716-a791-e88f343d62d9\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " Jul 12 14:50:59.202515 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202411 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-bundle\") pod \"309ebb07-a049-4716-a791-e88f343d62d9\" (UID: \"309ebb07-a049-4716-a791-e88f343d62d9\") " Jul 12 14:50:59.202515 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202438 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-bundle\") pod \"8507da11-9656-4b5f-9f36-105cb2fdae9a\" (UID: \"8507da11-9656-4b5f-9f36-105cb2fdae9a\") " Jul 12 14:50:59.202780 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202682 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-util\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.202780 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202703 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh2dr\" (UniqueName: \"kubernetes.io/projected/f48a30dc-8e56-4d30-a219-75661726d3c5-kube-api-access-bh2dr\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.202780 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202716 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f48a30dc-8e56-4d30-a219-75661726d3c5-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.203002 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.202976 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-bundle" (OuterVolumeSpecName: "bundle") pod "309ebb07-a049-4716-a791-e88f343d62d9" (UID: "309ebb07-a049-4716-a791-e88f343d62d9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:50:59.203394 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.203360 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-bundle" (OuterVolumeSpecName: "bundle") pod "8507da11-9656-4b5f-9f36-105cb2fdae9a" (UID: "8507da11-9656-4b5f-9f36-105cb2fdae9a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:50:59.204599 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.204573 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8507da11-9656-4b5f-9f36-105cb2fdae9a-kube-api-access-d6n94" (OuterVolumeSpecName: "kube-api-access-d6n94") pod "8507da11-9656-4b5f-9f36-105cb2fdae9a" (UID: "8507da11-9656-4b5f-9f36-105cb2fdae9a"). InnerVolumeSpecName "kube-api-access-d6n94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:50:59.204878 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.204858 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309ebb07-a049-4716-a791-e88f343d62d9-kube-api-access-nx49p" (OuterVolumeSpecName: "kube-api-access-nx49p") pod "309ebb07-a049-4716-a791-e88f343d62d9" (UID: "309ebb07-a049-4716-a791-e88f343d62d9"). InnerVolumeSpecName "kube-api-access-nx49p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:50:59.207324 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.207288 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-util" (OuterVolumeSpecName: "util") pod "309ebb07-a049-4716-a791-e88f343d62d9" (UID: "309ebb07-a049-4716-a791-e88f343d62d9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:50:59.208872 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.208852 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-util" (OuterVolumeSpecName: "util") pod "8507da11-9656-4b5f-9f36-105cb2fdae9a" (UID: "8507da11-9656-4b5f-9f36-105cb2fdae9a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:50:59.303523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.303427 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx49p\" (UniqueName: \"kubernetes.io/projected/309ebb07-a049-4716-a791-e88f343d62d9-kube-api-access-nx49p\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.303523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.303464 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.303523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.303474 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.303523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.303482 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8507da11-9656-4b5f-9f36-105cb2fdae9a-util\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.303523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.303489 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/309ebb07-a049-4716-a791-e88f343d62d9-util\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.303523 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.303496 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6n94\" (UniqueName: \"kubernetes.io/projected/8507da11-9656-4b5f-9f36-105cb2fdae9a-kube-api-access-d6n94\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:50:59.850023 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.849934 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:50:59.884932 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.884906 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" Jul 12 14:50:59.885314 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.884934 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7" event={"ID":"8507da11-9656-4b5f-9f36-105cb2fdae9a","Type":"ContainerDied","Data":"4d3725e919651d69434426376ec17625d0b9ec5e01ebfd3e353dea8a51e0f186"} Jul 12 14:50:59.885314 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.884981 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d3725e919651d69434426376ec17625d0b9ec5e01ebfd3e353dea8a51e0f186" Jul 12 14:50:59.886922 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.886890 2573 generic.go:356] "Generic (PLEG): container finished" podID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerID="0f690717df98259e2296006d453cfb2e34af009573744a9d4ec8222efabea3d3" exitCode=0 Jul 12 14:50:59.887081 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.886920 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" event={"ID":"cf2e9876-c6ef-4356-a884-14ec05b98a45","Type":"ContainerDied","Data":"0f690717df98259e2296006d453cfb2e34af009573744a9d4ec8222efabea3d3"} Jul 12 14:50:59.888821 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.888806 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" Jul 12 14:50:59.888821 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.888821 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb" event={"ID":"309ebb07-a049-4716-a791-e88f343d62d9","Type":"ContainerDied","Data":"ba3579bc831caddfeb46d4a16f9bac2b7b43317c8f0dd04a0c48f5007398f8bf"} Jul 12 14:50:59.889000 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.888850 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba3579bc831caddfeb46d4a16f9bac2b7b43317c8f0dd04a0c48f5007398f8bf" Jul 12 14:50:59.890750 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.890731 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" event={"ID":"f48a30dc-8e56-4d30-a219-75661726d3c5","Type":"ContainerDied","Data":"e712813cbdd7d8aba747ee0d7c2bc9cb43545970fd02fa6cc0c9f7c7100bbdd5"} Jul 12 14:50:59.890831 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.890766 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e712813cbdd7d8aba747ee0d7c2bc9cb43545970fd02fa6cc0c9f7c7100bbdd5" Jul 12 14:50:59.890831 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:50:59.890778 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49" Jul 12 14:51:01.027400 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.027374 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:51:01.119600 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.119572 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-util\") pod \"cf2e9876-c6ef-4356-a884-14ec05b98a45\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " Jul 12 14:51:01.119764 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.119663 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-478cc\" (UniqueName: \"kubernetes.io/projected/cf2e9876-c6ef-4356-a884-14ec05b98a45-kube-api-access-478cc\") pod \"cf2e9876-c6ef-4356-a884-14ec05b98a45\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " Jul 12 14:51:01.119764 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.119701 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-bundle\") pod \"cf2e9876-c6ef-4356-a884-14ec05b98a45\" (UID: \"cf2e9876-c6ef-4356-a884-14ec05b98a45\") " Jul 12 14:51:01.120294 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.120258 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-bundle" (OuterVolumeSpecName: "bundle") pod "cf2e9876-c6ef-4356-a884-14ec05b98a45" (UID: "cf2e9876-c6ef-4356-a884-14ec05b98a45"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:51:01.121929 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.121894 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf2e9876-c6ef-4356-a884-14ec05b98a45-kube-api-access-478cc" (OuterVolumeSpecName: "kube-api-access-478cc") pod "cf2e9876-c6ef-4356-a884-14ec05b98a45" (UID: "cf2e9876-c6ef-4356-a884-14ec05b98a45"). InnerVolumeSpecName "kube-api-access-478cc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:51:01.128474 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.128440 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-util" (OuterVolumeSpecName: "util") pod "cf2e9876-c6ef-4356-a884-14ec05b98a45" (UID: "cf2e9876-c6ef-4356-a884-14ec05b98a45"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 14:51:01.220747 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.220651 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-478cc\" (UniqueName: \"kubernetes.io/projected/cf2e9876-c6ef-4356-a884-14ec05b98a45-kube-api-access-478cc\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:51:01.220747 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.220687 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:51:01.220747 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.220700 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf2e9876-c6ef-4356-a884-14ec05b98a45-util\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:51:01.902472 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.902439 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" Jul 12 14:51:01.902644 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.902468 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7" event={"ID":"cf2e9876-c6ef-4356-a884-14ec05b98a45","Type":"ContainerDied","Data":"73cbe62fa7a56f301b2b8975d2ce31f7807889c59b7b679798e1893a598c3e8f"} Jul 12 14:51:01.902644 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:01.902516 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73cbe62fa7a56f301b2b8975d2ce31f7807889c59b7b679798e1893a598c3e8f" Jul 12 14:51:07.868695 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:07.868661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:12.073058 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:12.073000 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:14.512915 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:14.512885 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:14.685922 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:14.685842 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:15.922002 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:15.921970 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:19.077477 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:19.077443 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:19.483263 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:19.483231 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:19.874567 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:19.874538 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:22.510030 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:22.509962 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:22.918369 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:22.918338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:23.018374 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:23.018339 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:23.886699 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:23.886668 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:23.981444 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:23.981412 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:25.488855 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:25.488822 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:25.541667 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:25.541629 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:27.682677 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:27.682639 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:27.783256 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:27.783222 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:28.524890 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:28.524840 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:28.686263 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:28.686232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:31.076194 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:31.076158 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:31.132678 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:31.132630 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:36.527220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:36.527187 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c5d6dc4d7-2p55w"] Jul 12 14:51:36.645381 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:36.645349 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:36.744566 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:36.744527 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:36.803349 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:36.803266 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:39.529743 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:39.529708 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:39.635762 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:39.635729 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:42.115383 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:42.115343 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:42.214166 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:42.214127 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:44.939800 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:44.939763 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:45.039755 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:45.039722 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:48.470841 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:48.470804 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:48.572090 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:48.572052 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:56.650079 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:56.650043 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:56.750657 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:56.750620 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:57.296129 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:57.296096 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:57.395995 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:57.395963 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:59.898717 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:59.898684 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:51:59.999473 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:51:59.999433 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:01.547167 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.547119 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c5d6dc4d7-2p55w" podUID="e6d4e9f0-6886-4687-897d-b6cb1646d4fe" containerName="console" containerID="cri-o://bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab" gracePeriod=15 Jul 12 14:52:01.794959 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.794935 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:52:01.958271 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958223 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-trusted-ca-bundle\") pod \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " Jul 12 14:52:01.958271 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958270 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-oauth-serving-cert\") pod \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " Jul 12 14:52:01.958626 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958304 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-config\") pod \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " Jul 12 14:52:01.958626 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958325 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7wp5\" (UniqueName: \"kubernetes.io/projected/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-kube-api-access-f7wp5\") pod \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " Jul 12 14:52:01.958626 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958358 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-serving-cert\") pod \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " Jul 12 14:52:01.958626 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958384 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-oauth-config\") pod \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " Jul 12 14:52:01.958626 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958447 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-service-ca\") pod \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\" (UID: \"e6d4e9f0-6886-4687-897d-b6cb1646d4fe\") " Jul 12 14:52:01.958885 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958733 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-config" (OuterVolumeSpecName: "console-config") pod "e6d4e9f0-6886-4687-897d-b6cb1646d4fe" (UID: "e6d4e9f0-6886-4687-897d-b6cb1646d4fe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:52:01.958885 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958766 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e6d4e9f0-6886-4687-897d-b6cb1646d4fe" (UID: "e6d4e9f0-6886-4687-897d-b6cb1646d4fe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:52:01.958885 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958811 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e6d4e9f0-6886-4687-897d-b6cb1646d4fe" (UID: "e6d4e9f0-6886-4687-897d-b6cb1646d4fe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:52:01.958985 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.958907 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-service-ca" (OuterVolumeSpecName: "service-ca") pod "e6d4e9f0-6886-4687-897d-b6cb1646d4fe" (UID: "e6d4e9f0-6886-4687-897d-b6cb1646d4fe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 14:52:01.960526 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.960500 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-kube-api-access-f7wp5" (OuterVolumeSpecName: "kube-api-access-f7wp5") pod "e6d4e9f0-6886-4687-897d-b6cb1646d4fe" (UID: "e6d4e9f0-6886-4687-897d-b6cb1646d4fe"). InnerVolumeSpecName "kube-api-access-f7wp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:52:01.960909 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.960880 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e6d4e9f0-6886-4687-897d-b6cb1646d4fe" (UID: "e6d4e9f0-6886-4687-897d-b6cb1646d4fe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:52:01.961001 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:01.960904 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e6d4e9f0-6886-4687-897d-b6cb1646d4fe" (UID: "e6d4e9f0-6886-4687-897d-b6cb1646d4fe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:52:02.059476 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.059440 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-serving-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:02.059476 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.059471 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-oauth-config\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:02.059476 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.059482 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-service-ca\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:02.059738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.059491 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-trusted-ca-bundle\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:02.059738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.059500 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-oauth-serving-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:02.059738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.059508 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-console-config\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:02.059738 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.059515 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7wp5\" (UniqueName: \"kubernetes.io/projected/e6d4e9f0-6886-4687-897d-b6cb1646d4fe-kube-api-access-f7wp5\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:02.199331 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.199299 2573 generic.go:356] "Generic (PLEG): container finished" podID="e6d4e9f0-6886-4687-897d-b6cb1646d4fe" containerID="bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab" exitCode=0 Jul 12 14:52:02.199509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.199365 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c5d6dc4d7-2p55w" Jul 12 14:52:02.199509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.199385 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c5d6dc4d7-2p55w" event={"ID":"e6d4e9f0-6886-4687-897d-b6cb1646d4fe","Type":"ContainerDied","Data":"bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab"} Jul 12 14:52:02.199509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.199463 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c5d6dc4d7-2p55w" event={"ID":"e6d4e9f0-6886-4687-897d-b6cb1646d4fe","Type":"ContainerDied","Data":"4221a00a9881d596c92bc06cfb957c704a989beb69b2f2c345ca10e71a0a6ff3"} Jul 12 14:52:02.199509 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.199486 2573 scope.go:117] "RemoveContainer" containerID="bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab" Jul 12 14:52:02.208955 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.208934 2573 scope.go:117] "RemoveContainer" containerID="bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab" Jul 12 14:52:02.209302 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:52:02.209276 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab\": container with ID starting with bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab not found: ID does not exist" containerID="bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab" Jul 12 14:52:02.209408 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.209309 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab"} err="failed to get container status \"bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab\": rpc error: code = NotFound desc = could not find container \"bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab\": container with ID starting with bb1895165a4ad0ca288b65203500ad8c09c2e635aa4b55a95fb79981015081ab not found: ID does not exist" Jul 12 14:52:02.220173 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.220148 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c5d6dc4d7-2p55w"] Jul 12 14:52:02.224652 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.224620 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c5d6dc4d7-2p55w"] Jul 12 14:52:02.575744 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:02.575661 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d4e9f0-6886-4687-897d-b6cb1646d4fe" path="/var/lib/kubelet/pods/e6d4e9f0-6886-4687-897d-b6cb1646d4fe/volumes" Jul 12 14:52:03.130478 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:03.130445 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:03.236731 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:03.236671 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:06.497870 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:06.497834 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:06.559029 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:06.558983 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:11.900426 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:11.900386 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:12.164741 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:12.164646 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:15.092647 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:15.092608 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:15.193124 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:15.193086 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:27.936316 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:27.936284 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:28.053632 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.053581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:28.154079 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.154047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:28.735464 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.735430 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-78xlr"] Jul 12 14:52:28.735926 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.735900 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerName="pull" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.735986 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerName="pull" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736030 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerName="util" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736038 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerName="util" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736048 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309ebb07-a049-4716-a791-e88f343d62d9" containerName="pull" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736055 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="309ebb07-a049-4716-a791-e88f343d62d9" containerName="pull" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736062 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerName="extract" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736072 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerName="extract" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736087 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerName="util" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736094 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerName="util" Jul 12 14:52:28.736102 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736107 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d4e9f0-6886-4687-897d-b6cb1646d4fe" containerName="console" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736114 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d4e9f0-6886-4687-897d-b6cb1646d4fe" containerName="console" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736125 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309ebb07-a049-4716-a791-e88f343d62d9" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736132 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="309ebb07-a049-4716-a791-e88f343d62d9" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736144 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerName="pull" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736152 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerName="pull" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736162 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736170 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736184 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerName="pull" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736193 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerName="pull" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736208 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerName="util" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736216 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerName="util" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736231 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309ebb07-a049-4716-a791-e88f343d62d9" containerName="util" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736240 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="309ebb07-a049-4716-a791-e88f343d62d9" containerName="util" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736249 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736256 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736355 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d4e9f0-6886-4687-897d-b6cb1646d4fe" containerName="console" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736366 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f48a30dc-8e56-4d30-a219-75661726d3c5" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736373 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf2e9876-c6ef-4356-a884-14ec05b98a45" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736385 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8507da11-9656-4b5f-9f36-105cb2fdae9a" containerName="extract" Jul 12 14:52:28.736583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.736395 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="309ebb07-a049-4716-a791-e88f343d62d9" containerName="extract" Jul 12 14:52:28.739570 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.739549 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-78xlr" Jul 12 14:52:28.741533 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.741513 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 14:52:28.742109 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.742089 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6j976\"" Jul 12 14:52:28.742214 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.742093 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 14:52:28.747999 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.747978 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-78xlr"] Jul 12 14:52:28.767596 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.767570 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:28.768740 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.768719 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kzhz\" (UniqueName: \"kubernetes.io/projected/21f64de7-0a3d-4d44-a8b6-8925ce255730-kube-api-access-4kzhz\") pod \"authorino-58cd955675-78xlr\" (UID: \"21f64de7-0a3d-4d44-a8b6-8925ce255730\") " pod="kuadrant-system/authorino-58cd955675-78xlr" Jul 12 14:52:28.828882 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.828844 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:28.851650 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.851616 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jwmx"] Jul 12 14:52:28.854766 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.854743 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jwmx" Jul 12 14:52:28.864885 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.864855 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jwmx"] Jul 12 14:52:28.870167 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.870136 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kzhz\" (UniqueName: \"kubernetes.io/projected/21f64de7-0a3d-4d44-a8b6-8925ce255730-kube-api-access-4kzhz\") pod \"authorino-58cd955675-78xlr\" (UID: \"21f64de7-0a3d-4d44-a8b6-8925ce255730\") " pod="kuadrant-system/authorino-58cd955675-78xlr" Jul 12 14:52:28.881128 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.881098 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kzhz\" (UniqueName: \"kubernetes.io/projected/21f64de7-0a3d-4d44-a8b6-8925ce255730-kube-api-access-4kzhz\") pod \"authorino-58cd955675-78xlr\" (UID: \"21f64de7-0a3d-4d44-a8b6-8925ce255730\") " pod="kuadrant-system/authorino-58cd955675-78xlr" Jul 12 14:52:28.971455 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:28.971420 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxvpl\" (UniqueName: \"kubernetes.io/projected/7b7dad9a-20e2-40a2-8eea-e5bb375296e8-kube-api-access-pxvpl\") pod \"authorino-75d556bd77-9jwmx\" (UID: \"7b7dad9a-20e2-40a2-8eea-e5bb375296e8\") " pod="kuadrant-system/authorino-75d556bd77-9jwmx" Jul 12 14:52:29.049651 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.049561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-78xlr" Jul 12 14:52:29.072815 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.072784 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxvpl\" (UniqueName: \"kubernetes.io/projected/7b7dad9a-20e2-40a2-8eea-e5bb375296e8-kube-api-access-pxvpl\") pod \"authorino-75d556bd77-9jwmx\" (UID: \"7b7dad9a-20e2-40a2-8eea-e5bb375296e8\") " pod="kuadrant-system/authorino-75d556bd77-9jwmx" Jul 12 14:52:29.080083 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.080056 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxvpl\" (UniqueName: \"kubernetes.io/projected/7b7dad9a-20e2-40a2-8eea-e5bb375296e8-kube-api-access-pxvpl\") pod \"authorino-75d556bd77-9jwmx\" (UID: \"7b7dad9a-20e2-40a2-8eea-e5bb375296e8\") " pod="kuadrant-system/authorino-75d556bd77-9jwmx" Jul 12 14:52:29.170715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.170681 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jwmx" Jul 12 14:52:29.178055 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.178027 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-78xlr"] Jul 12 14:52:29.181318 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:52:29.181288 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21f64de7_0a3d_4d44_a8b6_8925ce255730.slice/crio-2bc4707c829d54f735de2c167191fc7a1ade0e65652317f01f733899924444bb WatchSource:0}: Error finding container 2bc4707c829d54f735de2c167191fc7a1ade0e65652317f01f733899924444bb: Status 404 returned error can't find the container with id 2bc4707c829d54f735de2c167191fc7a1ade0e65652317f01f733899924444bb Jul 12 14:52:29.182583 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.182562 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 14:52:29.298639 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.298613 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jwmx"] Jul 12 14:52:29.300165 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:52:29.300115 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b7dad9a_20e2_40a2_8eea_e5bb375296e8.slice/crio-b2e48b48d38436a47aca2f2cfa063e6f3738b46ba03143309631a849f65ccf2f WatchSource:0}: Error finding container b2e48b48d38436a47aca2f2cfa063e6f3738b46ba03143309631a849f65ccf2f: Status 404 returned error can't find the container with id b2e48b48d38436a47aca2f2cfa063e6f3738b46ba03143309631a849f65ccf2f Jul 12 14:52:29.334216 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.334179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-78xlr" event={"ID":"21f64de7-0a3d-4d44-a8b6-8925ce255730","Type":"ContainerStarted","Data":"2bc4707c829d54f735de2c167191fc7a1ade0e65652317f01f733899924444bb"} Jul 12 14:52:29.335288 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:29.335263 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jwmx" event={"ID":"7b7dad9a-20e2-40a2-8eea-e5bb375296e8","Type":"ContainerStarted","Data":"b2e48b48d38436a47aca2f2cfa063e6f3738b46ba03143309631a849f65ccf2f"} Jul 12 14:52:33.365525 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:33.365489 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-78xlr" event={"ID":"21f64de7-0a3d-4d44-a8b6-8925ce255730","Type":"ContainerStarted","Data":"444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a"} Jul 12 14:52:33.366822 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:33.366797 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jwmx" event={"ID":"7b7dad9a-20e2-40a2-8eea-e5bb375296e8","Type":"ContainerStarted","Data":"214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5"} Jul 12 14:52:33.380588 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:33.380539 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-78xlr" podStartSLOduration=1.947079741 podStartE2EDuration="5.38052627s" totalImagesPullingTime="3.433446529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:52:33.378196379 +0000 UTC m=+701.346079632" watchObservedRunningTime="2026-07-12 14:52:33.38052627 +0000 UTC m=+701.348409568" Jul 12 14:52:33.390715 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:33.390670 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9jwmx" podStartSLOduration=2.064453562 podStartE2EDuration="5.390657202s" totalImagesPullingTime="3.32620364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:52:33.390613012 +0000 UTC m=+701.358496267" watchObservedRunningTime="2026-07-12 14:52:33.390657202 +0000 UTC m=+701.358540452" Jul 12 14:52:33.412760 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:33.412729 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-78xlr"] Jul 12 14:52:35.377659 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:35.377598 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-78xlr" podUID="21f64de7-0a3d-4d44-a8b6-8925ce255730" containerName="authorino" containerID="cri-o://444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a" gracePeriod=30 Jul 12 14:52:35.634358 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:35.634287 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-78xlr" Jul 12 14:52:35.732916 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:35.732882 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kzhz\" (UniqueName: \"kubernetes.io/projected/21f64de7-0a3d-4d44-a8b6-8925ce255730-kube-api-access-4kzhz\") pod \"21f64de7-0a3d-4d44-a8b6-8925ce255730\" (UID: \"21f64de7-0a3d-4d44-a8b6-8925ce255730\") " Jul 12 14:52:35.734924 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:35.734899 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f64de7-0a3d-4d44-a8b6-8925ce255730-kube-api-access-4kzhz" (OuterVolumeSpecName: "kube-api-access-4kzhz") pod "21f64de7-0a3d-4d44-a8b6-8925ce255730" (UID: "21f64de7-0a3d-4d44-a8b6-8925ce255730"). InnerVolumeSpecName "kube-api-access-4kzhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:52:35.833543 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:35.833501 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kzhz\" (UniqueName: \"kubernetes.io/projected/21f64de7-0a3d-4d44-a8b6-8925ce255730-kube-api-access-4kzhz\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:52:36.383406 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.383369 2573 generic.go:356] "Generic (PLEG): container finished" podID="21f64de7-0a3d-4d44-a8b6-8925ce255730" containerID="444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a" exitCode=0 Jul 12 14:52:36.383836 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.383416 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-78xlr" Jul 12 14:52:36.383836 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.383430 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-78xlr" event={"ID":"21f64de7-0a3d-4d44-a8b6-8925ce255730","Type":"ContainerDied","Data":"444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a"} Jul 12 14:52:36.383836 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.383467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-78xlr" event={"ID":"21f64de7-0a3d-4d44-a8b6-8925ce255730","Type":"ContainerDied","Data":"2bc4707c829d54f735de2c167191fc7a1ade0e65652317f01f733899924444bb"} Jul 12 14:52:36.383836 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.383487 2573 scope.go:117] "RemoveContainer" containerID="444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a" Jul 12 14:52:36.392160 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.392139 2573 scope.go:117] "RemoveContainer" containerID="444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a" Jul 12 14:52:36.392400 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:52:36.392380 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a\": container with ID starting with 444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a not found: ID does not exist" containerID="444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a" Jul 12 14:52:36.392462 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.392414 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a"} err="failed to get container status \"444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a\": rpc error: code = NotFound desc = could not find container \"444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a\": container with ID starting with 444e451a76f818485891cf4f68753c22974c8d5f6001f0f1fc083c3d48667a9a not found: ID does not exist" Jul 12 14:52:36.402161 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.402140 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-78xlr"] Jul 12 14:52:36.406037 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.406003 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-78xlr"] Jul 12 14:52:36.576528 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:36.576499 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21f64de7-0a3d-4d44-a8b6-8925ce255730" path="/var/lib/kubelet/pods/21f64de7-0a3d-4d44-a8b6-8925ce255730/volumes" Jul 12 14:52:48.669243 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:48.669135 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:48.780644 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:48.780586 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:49.140573 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:49.140513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:56.793592 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:56.793561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:52:56.895290 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:52:56.895260 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:01.151276 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.151236 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:01.365205 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.365172 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-28d76"] Jul 12 14:53:01.365507 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.365495 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f64de7-0a3d-4d44-a8b6-8925ce255730" containerName="authorino" Jul 12 14:53:01.365561 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.365510 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f64de7-0a3d-4d44-a8b6-8925ce255730" containerName="authorino" Jul 12 14:53:01.365598 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.365574 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f64de7-0a3d-4d44-a8b6-8925ce255730" containerName="authorino" Jul 12 14:53:01.370152 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.370133 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-28d76" Jul 12 14:53:01.378854 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.378826 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-28d76"] Jul 12 14:53:01.401335 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.401257 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:01.560497 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.560460 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94cf8\" (UniqueName: \"kubernetes.io/projected/11b00595-6bf5-4494-ac5f-cc225ea760fb-kube-api-access-94cf8\") pod \"authorino-5d4b67c446-28d76\" (UID: \"11b00595-6bf5-4494-ac5f-cc225ea760fb\") " pod="kuadrant-system/authorino-5d4b67c446-28d76" Jul 12 14:53:01.583147 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.583110 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-28d76"] Jul 12 14:53:01.583384 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:53:01.583366 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-94cf8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-28d76" podUID="11b00595-6bf5-4494-ac5f-cc225ea760fb" Jul 12 14:53:01.608371 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.608321 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8499b6667f-9lxcm"] Jul 12 14:53:01.610504 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.610486 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8499b6667f-9lxcm" Jul 12 14:53:01.619831 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.619806 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8499b6667f-9lxcm"] Jul 12 14:53:01.661368 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.661279 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94cf8\" (UniqueName: \"kubernetes.io/projected/11b00595-6bf5-4494-ac5f-cc225ea760fb-kube-api-access-94cf8\") pod \"authorino-5d4b67c446-28d76\" (UID: \"11b00595-6bf5-4494-ac5f-cc225ea760fb\") " pod="kuadrant-system/authorino-5d4b67c446-28d76" Jul 12 14:53:01.661368 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.661365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z2nj\" (UniqueName: \"kubernetes.io/projected/e4e692b1-a5b7-48ad-8308-f93d045af3c3-kube-api-access-4z2nj\") pod \"authorino-8499b6667f-9lxcm\" (UID: \"e4e692b1-a5b7-48ad-8308-f93d045af3c3\") " pod="kuadrant-system/authorino-8499b6667f-9lxcm" Jul 12 14:53:01.669327 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.669305 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94cf8\" (UniqueName: \"kubernetes.io/projected/11b00595-6bf5-4494-ac5f-cc225ea760fb-kube-api-access-94cf8\") pod \"authorino-5d4b67c446-28d76\" (UID: \"11b00595-6bf5-4494-ac5f-cc225ea760fb\") " pod="kuadrant-system/authorino-5d4b67c446-28d76" Jul 12 14:53:01.761872 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.761839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z2nj\" (UniqueName: \"kubernetes.io/projected/e4e692b1-a5b7-48ad-8308-f93d045af3c3-kube-api-access-4z2nj\") pod \"authorino-8499b6667f-9lxcm\" (UID: \"e4e692b1-a5b7-48ad-8308-f93d045af3c3\") " pod="kuadrant-system/authorino-8499b6667f-9lxcm" Jul 12 14:53:01.769194 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.769169 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z2nj\" (UniqueName: \"kubernetes.io/projected/e4e692b1-a5b7-48ad-8308-f93d045af3c3-kube-api-access-4z2nj\") pod \"authorino-8499b6667f-9lxcm\" (UID: \"e4e692b1-a5b7-48ad-8308-f93d045af3c3\") " pod="kuadrant-system/authorino-8499b6667f-9lxcm" Jul 12 14:53:01.824421 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.824387 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8499b6667f-9lxcm"] Jul 12 14:53:01.824642 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.824630 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8499b6667f-9lxcm" Jul 12 14:53:01.851366 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.851333 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55f9b46db7-fh5m8"] Jul 12 14:53:01.853828 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.853801 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:01.855823 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.855796 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 12 14:53:01.862404 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.862373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl6c7\" (UniqueName: \"kubernetes.io/projected/3f452554-c088-4afb-9bce-355259f6f665-kube-api-access-tl6c7\") pod \"authorino-55f9b46db7-fh5m8\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:01.862541 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.862442 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f452554-c088-4afb-9bce-355259f6f665-tls-cert\") pod \"authorino-55f9b46db7-fh5m8\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:01.864604 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.864085 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55f9b46db7-fh5m8"] Jul 12 14:53:01.952766 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.952734 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8499b6667f-9lxcm"] Jul 12 14:53:01.954291 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:53:01.954264 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4e692b1_a5b7_48ad_8308_f93d045af3c3.slice/crio-2507626ebbe07f08c5db2429403586b1af54317035e01cffce32aaee680a2a1a WatchSource:0}: Error finding container 2507626ebbe07f08c5db2429403586b1af54317035e01cffce32aaee680a2a1a: Status 404 returned error can't find the container with id 2507626ebbe07f08c5db2429403586b1af54317035e01cffce32aaee680a2a1a Jul 12 14:53:01.963502 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.963477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl6c7\" (UniqueName: \"kubernetes.io/projected/3f452554-c088-4afb-9bce-355259f6f665-kube-api-access-tl6c7\") pod \"authorino-55f9b46db7-fh5m8\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:01.963591 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.963515 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f452554-c088-4afb-9bce-355259f6f665-tls-cert\") pod \"authorino-55f9b46db7-fh5m8\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:01.965790 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.965770 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f452554-c088-4afb-9bce-355259f6f665-tls-cert\") pod \"authorino-55f9b46db7-fh5m8\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:01.971606 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:01.971586 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl6c7\" (UniqueName: \"kubernetes.io/projected/3f452554-c088-4afb-9bce-355259f6f665-kube-api-access-tl6c7\") pod \"authorino-55f9b46db7-fh5m8\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:02.163765 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.163726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:53:02.287254 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.287229 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55f9b46db7-fh5m8"] Jul 12 14:53:02.289235 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:53:02.289211 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f452554_c088_4afb_9bce_355259f6f665.slice/crio-036304e3b6b6f67882e4e000e3bca0a04efcc5edfd081a202761760925ca39d1 WatchSource:0}: Error finding container 036304e3b6b6f67882e4e000e3bca0a04efcc5edfd081a202761760925ca39d1: Status 404 returned error can't find the container with id 036304e3b6b6f67882e4e000e3bca0a04efcc5edfd081a202761760925ca39d1 Jul 12 14:53:02.520101 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.519980 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8499b6667f-9lxcm" podUID="e4e692b1-a5b7-48ad-8308-f93d045af3c3" containerName="authorino" containerID="cri-o://57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a" gracePeriod=30 Jul 12 14:53:02.520101 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.519978 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8499b6667f-9lxcm" event={"ID":"e4e692b1-a5b7-48ad-8308-f93d045af3c3","Type":"ContainerStarted","Data":"57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a"} Jul 12 14:53:02.520321 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.520141 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8499b6667f-9lxcm" event={"ID":"e4e692b1-a5b7-48ad-8308-f93d045af3c3","Type":"ContainerStarted","Data":"2507626ebbe07f08c5db2429403586b1af54317035e01cffce32aaee680a2a1a"} Jul 12 14:53:02.521346 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.521326 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-28d76" Jul 12 14:53:02.521457 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.521326 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" event={"ID":"3f452554-c088-4afb-9bce-355259f6f665","Type":"ContainerStarted","Data":"036304e3b6b6f67882e4e000e3bca0a04efcc5edfd081a202761760925ca39d1"} Jul 12 14:53:02.527667 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.527651 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-28d76" Jul 12 14:53:02.534671 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.534634 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8499b6667f-9lxcm" podStartSLOduration=1.126966337 podStartE2EDuration="1.534621371s" totalImagesPullingTime="407.655034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:53:02.533299208 +0000 UTC m=+730.501182453" watchObservedRunningTime="2026-07-12 14:53:02.534621371 +0000 UTC m=+730.502504603" Jul 12 14:53:02.566782 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.566739 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94cf8\" (UniqueName: \"kubernetes.io/projected/11b00595-6bf5-4494-ac5f-cc225ea760fb-kube-api-access-94cf8\") pod \"11b00595-6bf5-4494-ac5f-cc225ea760fb\" (UID: \"11b00595-6bf5-4494-ac5f-cc225ea760fb\") " Jul 12 14:53:02.568871 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.568842 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b00595-6bf5-4494-ac5f-cc225ea760fb-kube-api-access-94cf8" (OuterVolumeSpecName: "kube-api-access-94cf8") pod "11b00595-6bf5-4494-ac5f-cc225ea760fb" (UID: "11b00595-6bf5-4494-ac5f-cc225ea760fb"). InnerVolumeSpecName "kube-api-access-94cf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:53:02.667352 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.667302 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94cf8\" (UniqueName: \"kubernetes.io/projected/11b00595-6bf5-4494-ac5f-cc225ea760fb-kube-api-access-94cf8\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:53:02.780386 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.780314 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8499b6667f-9lxcm" Jul 12 14:53:02.869429 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.869396 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z2nj\" (UniqueName: \"kubernetes.io/projected/e4e692b1-a5b7-48ad-8308-f93d045af3c3-kube-api-access-4z2nj\") pod \"e4e692b1-a5b7-48ad-8308-f93d045af3c3\" (UID: \"e4e692b1-a5b7-48ad-8308-f93d045af3c3\") " Jul 12 14:53:02.871419 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.871397 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4e692b1-a5b7-48ad-8308-f93d045af3c3-kube-api-access-4z2nj" (OuterVolumeSpecName: "kube-api-access-4z2nj") pod "e4e692b1-a5b7-48ad-8308-f93d045af3c3" (UID: "e4e692b1-a5b7-48ad-8308-f93d045af3c3"). InnerVolumeSpecName "kube-api-access-4z2nj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:53:02.970142 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:02.970103 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z2nj\" (UniqueName: \"kubernetes.io/projected/e4e692b1-a5b7-48ad-8308-f93d045af3c3-kube-api-access-4z2nj\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:53:03.527629 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.527599 2573 generic.go:356] "Generic (PLEG): container finished" podID="e4e692b1-a5b7-48ad-8308-f93d045af3c3" containerID="57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a" exitCode=0 Jul 12 14:53:03.528220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.527642 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8499b6667f-9lxcm" Jul 12 14:53:03.528220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.527690 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8499b6667f-9lxcm" event={"ID":"e4e692b1-a5b7-48ad-8308-f93d045af3c3","Type":"ContainerDied","Data":"57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a"} Jul 12 14:53:03.528220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.527731 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8499b6667f-9lxcm" event={"ID":"e4e692b1-a5b7-48ad-8308-f93d045af3c3","Type":"ContainerDied","Data":"2507626ebbe07f08c5db2429403586b1af54317035e01cffce32aaee680a2a1a"} Jul 12 14:53:03.528220 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.527754 2573 scope.go:117] "RemoveContainer" containerID="57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a" Jul 12 14:53:03.529400 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.529371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" event={"ID":"3f452554-c088-4afb-9bce-355259f6f665","Type":"ContainerStarted","Data":"3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575"} Jul 12 14:53:03.529487 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.529417 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-28d76" Jul 12 14:53:03.537172 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.537155 2573 scope.go:117] "RemoveContainer" containerID="57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a" Jul 12 14:53:03.537434 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:53:03.537416 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a\": container with ID starting with 57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a not found: ID does not exist" containerID="57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a" Jul 12 14:53:03.537485 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.537443 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a"} err="failed to get container status \"57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a\": rpc error: code = NotFound desc = could not find container \"57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a\": container with ID starting with 57bd3941d892d7cadc7e9c0a694cbfa913d8ac9850fa36c5584a80ad777c190a not found: ID does not exist" Jul 12 14:53:03.547023 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.546975 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" podStartSLOduration=2.223399337 podStartE2EDuration="2.546959471s" totalImagesPullingTime="323.560134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:53:03.545141779 +0000 UTC m=+731.513025024" watchObservedRunningTime="2026-07-12 14:53:03.546959471 +0000 UTC m=+731.514842724" Jul 12 14:53:03.562309 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.562280 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8499b6667f-9lxcm"] Jul 12 14:53:03.564655 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.564631 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8499b6667f-9lxcm"] Jul 12 14:53:03.573602 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.573573 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jwmx"] Jul 12 14:53:03.573804 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.573778 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9jwmx" podUID="7b7dad9a-20e2-40a2-8eea-e5bb375296e8" containerName="authorino" containerID="cri-o://214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5" gracePeriod=30 Jul 12 14:53:03.587357 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.587322 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-28d76"] Jul 12 14:53:03.592235 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.592208 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-28d76"] Jul 12 14:53:03.824872 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.824844 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jwmx" Jul 12 14:53:03.878284 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.878255 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxvpl\" (UniqueName: \"kubernetes.io/projected/7b7dad9a-20e2-40a2-8eea-e5bb375296e8-kube-api-access-pxvpl\") pod \"7b7dad9a-20e2-40a2-8eea-e5bb375296e8\" (UID: \"7b7dad9a-20e2-40a2-8eea-e5bb375296e8\") " Jul 12 14:53:03.880224 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.880195 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b7dad9a-20e2-40a2-8eea-e5bb375296e8-kube-api-access-pxvpl" (OuterVolumeSpecName: "kube-api-access-pxvpl") pod "7b7dad9a-20e2-40a2-8eea-e5bb375296e8" (UID: "7b7dad9a-20e2-40a2-8eea-e5bb375296e8"). InnerVolumeSpecName "kube-api-access-pxvpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:53:03.978945 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:03.978901 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxvpl\" (UniqueName: \"kubernetes.io/projected/7b7dad9a-20e2-40a2-8eea-e5bb375296e8-kube-api-access-pxvpl\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:53:04.534539 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.534509 2573 generic.go:356] "Generic (PLEG): container finished" podID="7b7dad9a-20e2-40a2-8eea-e5bb375296e8" containerID="214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5" exitCode=0 Jul 12 14:53:04.534991 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.534556 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jwmx" Jul 12 14:53:04.534991 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.534594 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jwmx" event={"ID":"7b7dad9a-20e2-40a2-8eea-e5bb375296e8","Type":"ContainerDied","Data":"214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5"} Jul 12 14:53:04.534991 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.534650 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jwmx" event={"ID":"7b7dad9a-20e2-40a2-8eea-e5bb375296e8","Type":"ContainerDied","Data":"b2e48b48d38436a47aca2f2cfa063e6f3738b46ba03143309631a849f65ccf2f"} Jul 12 14:53:04.534991 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.534677 2573 scope.go:117] "RemoveContainer" containerID="214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5" Jul 12 14:53:04.543588 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.543554 2573 scope.go:117] "RemoveContainer" containerID="214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5" Jul 12 14:53:04.543843 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:53:04.543823 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5\": container with ID starting with 214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5 not found: ID does not exist" containerID="214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5" Jul 12 14:53:04.543934 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.543851 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5"} err="failed to get container status \"214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5\": rpc error: code = NotFound desc = could not find container \"214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5\": container with ID starting with 214ee32c1b44aead3db0d76d770b9d5aa22b288d12f385c38e8ca9d5c452ceb5 not found: ID does not exist" Jul 12 14:53:04.555312 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.555287 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jwmx"] Jul 12 14:53:04.558482 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.558456 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jwmx"] Jul 12 14:53:04.576375 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.576345 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b00595-6bf5-4494-ac5f-cc225ea760fb" path="/var/lib/kubelet/pods/11b00595-6bf5-4494-ac5f-cc225ea760fb/volumes" Jul 12 14:53:04.576582 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.576569 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b7dad9a-20e2-40a2-8eea-e5bb375296e8" path="/var/lib/kubelet/pods/7b7dad9a-20e2-40a2-8eea-e5bb375296e8/volumes" Jul 12 14:53:04.576912 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:04.576900 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4e692b1-a5b7-48ad-8308-f93d045af3c3" path="/var/lib/kubelet/pods/e4e692b1-a5b7-48ad-8308-f93d045af3c3/volumes" Jul 12 14:53:07.797728 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:07.797695 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:07.895009 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:07.894980 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:07.946161 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:07.946131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:08.072708 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:08.072622 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:10.821457 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:10.821423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:10.918106 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:10.918071 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:43.311282 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:43.311204 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:43.749428 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:43.749398 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:44.055051 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:44.054929 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:44.171754 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:44.171712 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:44.264906 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:44.264838 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:54.149374 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:54.149329 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:54.246111 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:54.246075 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:53:59.422315 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:53:59.422274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:54:10.785446 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.785410 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b"] Jul 12 14:54:10.785834 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.785727 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7dad9a-20e2-40a2-8eea-e5bb375296e8" containerName="authorino" Jul 12 14:54:10.785834 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.785738 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7dad9a-20e2-40a2-8eea-e5bb375296e8" containerName="authorino" Jul 12 14:54:10.785834 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.785753 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4e692b1-a5b7-48ad-8308-f93d045af3c3" containerName="authorino" Jul 12 14:54:10.785834 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.785759 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4e692b1-a5b7-48ad-8308-f93d045af3c3" containerName="authorino" Jul 12 14:54:10.785834 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.785810 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7dad9a-20e2-40a2-8eea-e5bb375296e8" containerName="authorino" Jul 12 14:54:10.785834 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.785818 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4e692b1-a5b7-48ad-8308-f93d045af3c3" containerName="authorino" Jul 12 14:54:10.790113 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.790096 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.792793 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.792769 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 14:54:10.792793 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.792776 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nfnm5\"" Jul 12 14:54:10.792970 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.792776 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 12 14:54:10.792970 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.792845 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 14:54:10.798614 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.798588 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b"] Jul 12 14:54:10.875454 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.875423 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb71cc5-45e8-424f-b595-96dc8ae51606-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.875454 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.875461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.875676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.875493 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.875676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.875560 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.875676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.875581 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.875676 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.875600 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvlph\" (UniqueName: \"kubernetes.io/projected/6bb71cc5-45e8-424f-b595-96dc8ae51606-kube-api-access-tvlph\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.894626 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.894597 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:54:10.976533 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.976485 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.976533 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.976541 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.976804 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.976645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvlph\" (UniqueName: \"kubernetes.io/projected/6bb71cc5-45e8-424f-b595-96dc8ae51606-kube-api-access-tvlph\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.976804 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.976696 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb71cc5-45e8-424f-b595-96dc8ae51606-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.976804 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.976723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.976804 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.976761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.977059 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.976991 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.977189 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.977086 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.977189 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.977131 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.978692 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.978668 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bb71cc5-45e8-424f-b595-96dc8ae51606-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.979055 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.979034 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb71cc5-45e8-424f-b595-96dc8ae51606-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:10.984970 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:10.984950 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvlph\" (UniqueName: \"kubernetes.io/projected/6bb71cc5-45e8-424f-b595-96dc8ae51606-kube-api-access-tvlph\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8rm5b\" (UID: \"6bb71cc5-45e8-424f-b595-96dc8ae51606\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:11.101124 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:11.101009 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:11.228622 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:11.228598 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b"] Jul 12 14:54:11.230656 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:54:11.230628 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb71cc5_45e8_424f_b595_96dc8ae51606.slice/crio-807f259094af3ee1d7815d8b6a2fda06b08de64f897f85787ca4ef25f2bee5c0 WatchSource:0}: Error finding container 807f259094af3ee1d7815d8b6a2fda06b08de64f897f85787ca4ef25f2bee5c0: Status 404 returned error can't find the container with id 807f259094af3ee1d7815d8b6a2fda06b08de64f897f85787ca4ef25f2bee5c0 Jul 12 14:54:11.868193 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:11.868157 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" event={"ID":"6bb71cc5-45e8-424f-b595-96dc8ae51606","Type":"ContainerStarted","Data":"807f259094af3ee1d7815d8b6a2fda06b08de64f897f85787ca4ef25f2bee5c0"} Jul 12 14:54:12.880627 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:12.880589 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" event={"ID":"6bb71cc5-45e8-424f-b595-96dc8ae51606","Type":"ContainerStarted","Data":"fe43caec314afb26116049782a213ad1061bb26d45139aa7b1b24f3ef3d08565"} Jul 12 14:54:12.896750 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:12.896705 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" podStartSLOduration=1.870772165 podStartE2EDuration="2.896690247s" totalImagesPullingTime="1.025918082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:54:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:54:12.895572631 +0000 UTC m=+800.863455884" watchObservedRunningTime="2026-07-12 14:54:12.896690247 +0000 UTC m=+800.864573499" Jul 12 14:54:21.101232 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:21.101128 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:21.101232 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:21.101208 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:21.113555 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:21.113521 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:21.935429 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:21.935399 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8rm5b" Jul 12 14:54:33.016080 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:33.016047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:54:43.691452 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.691417 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc"] Jul 12 14:54:43.702474 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.702435 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.703875 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.703851 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc"] Jul 12 14:54:43.704776 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.704757 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 12 14:54:43.726949 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.726921 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:54:43.758237 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.758201 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv87l\" (UniqueName: \"kubernetes.io/projected/96b99278-616c-4b2a-9693-48505b0349d8-kube-api-access-xv87l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.758400 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.758280 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.758400 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.758311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.758400 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.758335 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.758400 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.758386 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.758587 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.758455 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b99278-616c-4b2a-9693-48505b0349d8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859055 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv87l\" (UniqueName: \"kubernetes.io/projected/96b99278-616c-4b2a-9693-48505b0349d8-kube-api-access-xv87l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859258 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859068 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859258 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859258 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859258 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859143 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859258 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b99278-616c-4b2a-9693-48505b0349d8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859496 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859474 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859556 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859540 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.859641 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.859623 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.861379 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.861356 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96b99278-616c-4b2a-9693-48505b0349d8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.861654 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.861634 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b99278-616c-4b2a-9693-48505b0349d8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:43.865934 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:43.865913 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv87l\" (UniqueName: \"kubernetes.io/projected/96b99278-616c-4b2a-9693-48505b0349d8-kube-api-access-xv87l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc\" (UID: \"96b99278-616c-4b2a-9693-48505b0349d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:44.012909 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:44.012813 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:44.146999 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:44.146962 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc"] Jul 12 14:54:44.148505 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:54:44.148469 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b99278_616c_4b2a_9693_48505b0349d8.slice/crio-47389ff2364d79b2ebde192123be8788553e7e2d838f7e24660b0d9d0aa8126f WatchSource:0}: Error finding container 47389ff2364d79b2ebde192123be8788553e7e2d838f7e24660b0d9d0aa8126f: Status 404 returned error can't find the container with id 47389ff2364d79b2ebde192123be8788553e7e2d838f7e24660b0d9d0aa8126f Jul 12 14:54:45.044800 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:45.044743 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" event={"ID":"96b99278-616c-4b2a-9693-48505b0349d8","Type":"ContainerStarted","Data":"cad4c170399a7970682d4696fd9802fdcea8b03764ff6d5c6bb09bdcfc349324"} Jul 12 14:54:45.045297 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:45.044835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" event={"ID":"96b99278-616c-4b2a-9693-48505b0349d8","Type":"ContainerStarted","Data":"47389ff2364d79b2ebde192123be8788553e7e2d838f7e24660b0d9d0aa8126f"} Jul 12 14:54:45.062963 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:45.062906 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" podStartSLOduration=1.877204097 podStartE2EDuration="2.062889799s" totalImagesPullingTime="185.685702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:54:45.062079737 +0000 UTC m=+833.029962990" watchObservedRunningTime="2026-07-12 14:54:45.062889799 +0000 UTC m=+833.030773052" Jul 12 14:54:47.286468 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:47.286419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:54:54.013314 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:54.013268 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:54.013729 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:54.013349 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:54.025551 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:54.025520 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:54:54.101414 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:54:54.101387 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc" Jul 12 14:55:01.143758 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:55:01.143724 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:55:45.171635 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:55:45.171542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:55:52.520907 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:55:52.520875 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:55:52.523476 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:55:52.523454 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 14:55:52.527446 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:55:52.527427 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:55:52.529371 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:55:52.529344 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 14:56:01.985485 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:56:01.985450 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:56:08.274842 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:56:08.274806 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:56:08.330215 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:56:08.330178 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:57:01.015973 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:01.015943 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:57:02.978122 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:02.978083 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84dcbdc54b-scqgq"] Jul 12 14:57:02.981446 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:02.981426 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:02.999443 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:02.999412 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84dcbdc54b-scqgq"] Jul 12 14:57:03.075386 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.075348 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmz85\" (UniqueName: \"kubernetes.io/projected/a1442e75-527f-4a31-bc6d-ea14bf4f6a13-kube-api-access-rmz85\") pod \"authorino-84dcbdc54b-scqgq\" (UID: \"a1442e75-527f-4a31-bc6d-ea14bf4f6a13\") " pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:03.075386 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.075398 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1442e75-527f-4a31-bc6d-ea14bf4f6a13-tls-cert\") pod \"authorino-84dcbdc54b-scqgq\" (UID: \"a1442e75-527f-4a31-bc6d-ea14bf4f6a13\") " pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:03.176890 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.176838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmz85\" (UniqueName: \"kubernetes.io/projected/a1442e75-527f-4a31-bc6d-ea14bf4f6a13-kube-api-access-rmz85\") pod \"authorino-84dcbdc54b-scqgq\" (UID: \"a1442e75-527f-4a31-bc6d-ea14bf4f6a13\") " pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:03.176890 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.176896 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1442e75-527f-4a31-bc6d-ea14bf4f6a13-tls-cert\") pod \"authorino-84dcbdc54b-scqgq\" (UID: \"a1442e75-527f-4a31-bc6d-ea14bf4f6a13\") " pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:03.179385 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.179362 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1442e75-527f-4a31-bc6d-ea14bf4f6a13-tls-cert\") pod \"authorino-84dcbdc54b-scqgq\" (UID: \"a1442e75-527f-4a31-bc6d-ea14bf4f6a13\") " pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:03.183849 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.183821 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmz85\" (UniqueName: \"kubernetes.io/projected/a1442e75-527f-4a31-bc6d-ea14bf4f6a13-kube-api-access-rmz85\") pod \"authorino-84dcbdc54b-scqgq\" (UID: \"a1442e75-527f-4a31-bc6d-ea14bf4f6a13\") " pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:03.290417 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.290308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dcbdc54b-scqgq" Jul 12 14:57:03.329850 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.329799 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:57:03.428581 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.428544 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:57:03.433855 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.433826 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84dcbdc54b-scqgq"] Jul 12 14:57:03.435718 ip-10-0-142-78 kubenswrapper[2573]: W0712 14:57:03.435684 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1442e75_527f_4a31_bc6d_ea14bf4f6a13.slice/crio-7fdece9e6e7cab24ec97dcbb29da7fdc45fd9517127e928b9c0c4d738162fc52 WatchSource:0}: Error finding container 7fdece9e6e7cab24ec97dcbb29da7fdc45fd9517127e928b9c0c4d738162fc52: Status 404 returned error can't find the container with id 7fdece9e6e7cab24ec97dcbb29da7fdc45fd9517127e928b9c0c4d738162fc52 Jul 12 14:57:03.739832 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:03.739798 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84dcbdc54b-scqgq" event={"ID":"a1442e75-527f-4a31-bc6d-ea14bf4f6a13","Type":"ContainerStarted","Data":"7fdece9e6e7cab24ec97dcbb29da7fdc45fd9517127e928b9c0c4d738162fc52"} Jul 12 14:57:04.416429 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:04.416393 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:57:04.523936 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:04.523885 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:57:04.747910 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:04.747788 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84dcbdc54b-scqgq" event={"ID":"a1442e75-527f-4a31-bc6d-ea14bf4f6a13","Type":"ContainerStarted","Data":"dd86304359d165b780d2d06f6dc7e1d76f5c6f4c4c754569e1e734371920e6f4"} Jul 12 14:57:04.766344 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:04.766286 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84dcbdc54b-scqgq" podStartSLOduration=2.398037389 podStartE2EDuration="2.766265565s" totalImagesPullingTime="368.228176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 14:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 14:57:04.764340206 +0000 UTC m=+972.732223459" watchObservedRunningTime="2026-07-12 14:57:04.766265565 +0000 UTC m=+972.734148817" Jul 12 14:57:04.797845 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:04.797811 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55f9b46db7-fh5m8"] Jul 12 14:57:04.798043 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:04.798004 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" podUID="3f452554-c088-4afb-9bce-355259f6f665" containerName="authorino" containerID="cri-o://3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575" gracePeriod=30 Jul 12 14:57:05.060571 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.060541 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:57:05.100701 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.100668 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl6c7\" (UniqueName: \"kubernetes.io/projected/3f452554-c088-4afb-9bce-355259f6f665-kube-api-access-tl6c7\") pod \"3f452554-c088-4afb-9bce-355259f6f665\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " Jul 12 14:57:05.100892 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.100733 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f452554-c088-4afb-9bce-355259f6f665-tls-cert\") pod \"3f452554-c088-4afb-9bce-355259f6f665\" (UID: \"3f452554-c088-4afb-9bce-355259f6f665\") " Jul 12 14:57:05.102845 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.102803 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f452554-c088-4afb-9bce-355259f6f665-kube-api-access-tl6c7" (OuterVolumeSpecName: "kube-api-access-tl6c7") pod "3f452554-c088-4afb-9bce-355259f6f665" (UID: "3f452554-c088-4afb-9bce-355259f6f665"). InnerVolumeSpecName "kube-api-access-tl6c7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 14:57:05.111313 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.111280 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f452554-c088-4afb-9bce-355259f6f665-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3f452554-c088-4afb-9bce-355259f6f665" (UID: "3f452554-c088-4afb-9bce-355259f6f665"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 14:57:05.202212 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.202160 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl6c7\" (UniqueName: \"kubernetes.io/projected/3f452554-c088-4afb-9bce-355259f6f665-kube-api-access-tl6c7\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:57:05.202212 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.202198 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3f452554-c088-4afb-9bce-355259f6f665-tls-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 14:57:05.754557 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.754519 2573 generic.go:356] "Generic (PLEG): container finished" podID="3f452554-c088-4afb-9bce-355259f6f665" containerID="3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575" exitCode=0 Jul 12 14:57:05.754975 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.754569 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" Jul 12 14:57:05.754975 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.754609 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" event={"ID":"3f452554-c088-4afb-9bce-355259f6f665","Type":"ContainerDied","Data":"3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575"} Jul 12 14:57:05.754975 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.754667 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55f9b46db7-fh5m8" event={"ID":"3f452554-c088-4afb-9bce-355259f6f665","Type":"ContainerDied","Data":"036304e3b6b6f67882e4e000e3bca0a04efcc5edfd081a202761760925ca39d1"} Jul 12 14:57:05.754975 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.754694 2573 scope.go:117] "RemoveContainer" containerID="3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575" Jul 12 14:57:05.764651 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.764627 2573 scope.go:117] "RemoveContainer" containerID="3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575" Jul 12 14:57:05.764965 ip-10-0-142-78 kubenswrapper[2573]: E0712 14:57:05.764945 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575\": container with ID starting with 3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575 not found: ID does not exist" containerID="3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575" Jul 12 14:57:05.765050 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.764974 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575"} err="failed to get container status \"3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575\": rpc error: code = NotFound desc = could not find container \"3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575\": container with ID starting with 3659f79c0bb9596ab91f95c938a2495bbda4e94485d0dd9fca0927c324a8a575 not found: ID does not exist" Jul 12 14:57:05.777918 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.777886 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55f9b46db7-fh5m8"] Jul 12 14:57:05.783625 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:05.783596 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55f9b46db7-fh5m8"] Jul 12 14:57:06.576259 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:06.576231 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f452554-c088-4afb-9bce-355259f6f665" path="/var/lib/kubelet/pods/3f452554-c088-4afb-9bce-355259f6f665/volumes" Jul 12 14:57:40.776422 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:40.776386 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:57:40.877395 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:57:40.877363 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:02.211541 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:02.211502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:13.944691 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:13.944651 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:15.748094 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:15.748064 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:17.646710 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:17.646678 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:17.747590 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:17.747555 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:56.056714 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:56.056627 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:56.165853 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:56.165820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:58:56.256846 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:58:56.256813 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:59:01.999858 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:59:01.999820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:59:02.113099 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:59:02.113066 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:59:02.200744 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:59:02.200708 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:59:02.347741 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:59:02.347661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:59:07.945223 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:59:07.945170 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 14:59:08.046188 ip-10-0-142-78 kubenswrapper[2573]: I0712 14:59:08.046143 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:00:52.552275 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:00:52.552244 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:00:52.556414 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:00:52.556390 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:00:52.558755 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:00:52.558730 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:00:52.562715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:00:52.562695 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:03:17.252074 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:03:17.251963 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:04:55.336432 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:04:55.336341 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:05:36.326278 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:05:36.326242 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:05:52.590837 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:05:52.590805 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:05:52.594615 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:05:52.594593 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:05:52.597205 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:05:52.597187 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:05:52.600828 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:05:52.600813 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:06:52.118209 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:06:52.118171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:06:52.217532 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:06:52.217498 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:09:55.081695 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:09:55.081662 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:09:56.755154 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:09:56.755101 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:01.337809 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.337774 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:01.410296 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.410268 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:01.413207 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.413177 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4"] Jul 12 15:10:01.413729 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.413709 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f452554-c088-4afb-9bce-355259f6f665" containerName="authorino" Jul 12 15:10:01.413802 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.413732 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f452554-c088-4afb-9bce-355259f6f665" containerName="authorino" Jul 12 15:10:01.413857 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.413826 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f452554-c088-4afb-9bce-355259f6f665" containerName="authorino" Jul 12 15:10:01.417232 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.417211 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.419442 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.419421 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-91d5d23f-openshift-default-dockercfg-4n5qp\"" Jul 12 15:10:01.436667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.436646 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4"] Jul 12 15:10:01.465464 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465442 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-envoy\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465577 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465474 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbc51022-6147-481c-871b-556a23422245-istiod-ca-cert\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465577 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465494 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-istio-token\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465577 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465513 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-certs\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465708 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465590 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-data\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465708 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-socket\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465708 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465661 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-credential-socket\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465708 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465695 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qjkz\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-kube-api-access-9qjkz\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.465834 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.465720 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbc51022-6147-481c-871b-556a23422245-istio-podinfo\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.544267 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.544238 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:01.567119 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567090 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-data\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567220 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567148 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-socket\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567220 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567167 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-credential-socket\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567220 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567194 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qjkz\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-kube-api-access-9qjkz\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567220 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbc51022-6147-481c-871b-556a23422245-istio-podinfo\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567429 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567242 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-envoy\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567429 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567267 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbc51022-6147-481c-871b-556a23422245-istiod-ca-cert\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567429 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567293 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-istio-token\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567429 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567323 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-certs\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567642 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567516 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-data\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567642 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567594 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-credential-socket\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567760 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567682 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-certs\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567823 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567798 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-socket\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.567978 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.567955 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbc51022-6147-481c-871b-556a23422245-istiod-ca-cert\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.569373 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.569354 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-envoy\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.569604 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.569589 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbc51022-6147-481c-871b-556a23422245-istio-podinfo\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.574090 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.574065 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-istio-token\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.574180 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.574162 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qjkz\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-kube-api-access-9qjkz\") pod \"e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.728704 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.728681 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:01.865251 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.861245 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4"] Jul 12 15:10:01.866918 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:10:01.866884 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc51022_6147_481c_871b_556a23422245.slice/crio-44a35faacddcbacaebefe16bdd57efb124521b9e21297fb48fa6266b121cfa91 WatchSource:0}: Error finding container 44a35faacddcbacaebefe16bdd57efb124521b9e21297fb48fa6266b121cfa91: Status 404 returned error can't find the container with id 44a35faacddcbacaebefe16bdd57efb124521b9e21297fb48fa6266b121cfa91 Jul 12 15:10:01.869047 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.868992 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:01.869137 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.869080 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:01.869137 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:01.869116 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:02.545972 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.545938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:02.622772 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.622740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:02.640920 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.640892 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" event={"ID":"bbc51022-6147-481c-871b-556a23422245","Type":"ContainerStarted","Data":"613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d"} Jul 12 15:10:02.641044 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.640948 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" event={"ID":"bbc51022-6147-481c-871b-556a23422245","Type":"ContainerStarted","Data":"44a35faacddcbacaebefe16bdd57efb124521b9e21297fb48fa6266b121cfa91"} Jul 12 15:10:02.660329 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.660290 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" podStartSLOduration=1.6602754370000001 podStartE2EDuration="1.660275437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:10:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:02.65884404 +0000 UTC m=+1750.626727295" watchObservedRunningTime="2026-07-12 15:10:02.660275437 +0000 UTC m=+1750.628158690" Jul 12 15:10:02.683926 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.683903 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:02.729330 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.729308 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:02.734477 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.734458 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:02.983301 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:02.983269 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:03.207950 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:03.207918 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:03.305696 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:03.305619 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:03.645833 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:03.645807 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:03.646785 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:03.646768 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:05.336281 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:05.336252 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:07.698519 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:07.698484 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:07.878670 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:07.878637 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4"] Jul 12 15:10:07.878883 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:07.878851 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" podUID="bbc51022-6147-481c-871b-556a23422245" containerName="istio-proxy" containerID="cri-o://613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d" gracePeriod=30 Jul 12 15:10:07.899834 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:07.899797 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:07.995374 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:07.995304 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:08.115861 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:08.115831 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:09.271730 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:09.271690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:11.063104 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:11.063052 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:13.224063 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.224036 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:13.375279 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375242 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-istio-token\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375423 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375292 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-credential-socket\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375423 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375344 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qjkz\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-kube-api-access-9qjkz\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375423 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375372 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbc51022-6147-481c-871b-556a23422245-istiod-ca-cert\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375423 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375399 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-data\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375637 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375439 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbc51022-6147-481c-871b-556a23422245-istio-podinfo\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375637 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375510 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-envoy\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375637 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375556 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-certs\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375637 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375579 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-socket\") pod \"bbc51022-6147-481c-871b-556a23422245\" (UID: \"bbc51022-6147-481c-871b-556a23422245\") " Jul 12 15:10:13.375847 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375650 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:13.375903 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375786 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbc51022-6147-481c-871b-556a23422245-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:10:13.375957 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375852 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:13.375957 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.375872 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.376097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.376066 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:13.376347 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.376318 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-data" (OuterVolumeSpecName: "istio-data") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:13.377629 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.377604 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:13.377742 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.377706 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-kube-api-access-9qjkz" (OuterVolumeSpecName: "kube-api-access-9qjkz") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "kube-api-access-9qjkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:10:13.377742 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.377706 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-istio-token" (OuterVolumeSpecName: "istio-token") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:10:13.377817 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.377770 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbc51022-6147-481c-871b-556a23422245-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bbc51022-6147-481c-871b-556a23422245" (UID: "bbc51022-6147-481c-871b-556a23422245"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:10:13.476665 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476641 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.476665 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476663 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.476799 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476672 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.476799 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476685 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qjkz\" (UniqueName: \"kubernetes.io/projected/bbc51022-6147-481c-871b-556a23422245-kube-api-access-9qjkz\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.476799 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476700 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbc51022-6147-481c-871b-556a23422245-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.476799 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476708 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.476799 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476716 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbc51022-6147-481c-871b-556a23422245-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.476799 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.476723 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbc51022-6147-481c-871b-556a23422245-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:13.661551 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.661488 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:13.702132 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.702104 2573 generic.go:356] "Generic (PLEG): container finished" podID="bbc51022-6147-481c-871b-556a23422245" containerID="613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d" exitCode=0 Jul 12 15:10:13.702237 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.702142 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" Jul 12 15:10:13.702237 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.702186 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" event={"ID":"bbc51022-6147-481c-871b-556a23422245","Type":"ContainerDied","Data":"613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d"} Jul 12 15:10:13.702319 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.702236 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4" event={"ID":"bbc51022-6147-481c-871b-556a23422245","Type":"ContainerDied","Data":"44a35faacddcbacaebefe16bdd57efb124521b9e21297fb48fa6266b121cfa91"} Jul 12 15:10:13.702319 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.702260 2573 scope.go:117] "RemoveContainer" containerID="613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d" Jul 12 15:10:13.711434 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.711390 2573 scope.go:117] "RemoveContainer" containerID="613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d" Jul 12 15:10:13.711651 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:10:13.711635 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d\": container with ID starting with 613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d not found: ID does not exist" containerID="613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d" Jul 12 15:10:13.711704 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.711661 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d"} err="failed to get container status \"613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d\": rpc error: code = NotFound desc = could not find container \"613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d\": container with ID starting with 613906672fba63d392c629f99091b4da2efa511def2edf6b0dcf7115ef23a88d not found: ID does not exist" Jul 12 15:10:13.725557 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.725493 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4"] Jul 12 15:10:13.730140 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.730111 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:13.730469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.730453 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-91d5d23f-openshift-default-74d478995f-5v6q4"] Jul 12 15:10:13.816355 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.816330 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp"] Jul 12 15:10:13.816816 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.816795 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc51022-6147-481c-871b-556a23422245" containerName="istio-proxy" Jul 12 15:10:13.816816 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.816815 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc51022-6147-481c-871b-556a23422245" containerName="istio-proxy" Jul 12 15:10:13.816991 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.816921 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc51022-6147-481c-871b-556a23422245" containerName="istio-proxy" Jul 12 15:10:13.821249 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.821229 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.823563 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.823544 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-61e6b661-openshift-default-dockercfg-kxxqx\"" Jul 12 15:10:13.830655 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.830635 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp"] Jul 12 15:10:13.855498 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.855460 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:13.878708 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.878683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istiod-ca-cert\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.878802 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.878715 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcdxq\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-kube-api-access-vcdxq\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.878802 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.878738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-podinfo\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.878913 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.878823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-socket\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.878913 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.878869 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-data\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.879047 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.878948 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-token\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.879047 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.878980 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-credential-socket\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.879147 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.879088 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-envoy\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.879147 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.879140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-certs\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.904385 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.904362 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:13.979619 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-envoy\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.979619 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979591 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-certs\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.979619 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979612 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istiod-ca-cert\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.979826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979629 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcdxq\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-kube-api-access-vcdxq\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.979826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-podinfo\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.979826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-socket\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.979826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979696 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-data\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.979826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979737 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-token\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.980117 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.979896 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-credential-socket\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.980117 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.980004 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-certs\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.980228 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.980120 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-socket\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.980283 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.980249 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-credential-socket\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.980352 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.980319 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-data\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.980405 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.980371 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istiod-ca-cert\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.981739 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.981721 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-envoy\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.981982 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.981963 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-podinfo\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.987060 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.987036 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-token\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:13.987206 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:13.987187 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcdxq\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-kube-api-access-vcdxq\") pod \"e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:14.134296 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.134274 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:14.264681 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.264652 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp"] Jul 12 15:10:14.266562 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:10:14.266533 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d7ed85_7159_4f68_a543_881c9e5dbd64.slice/crio-7a37d664a8e38aa020896d08c7497f20a8ae17ebd87351c7a939f6d5f5bc6768 WatchSource:0}: Error finding container 7a37d664a8e38aa020896d08c7497f20a8ae17ebd87351c7a939f6d5f5bc6768: Status 404 returned error can't find the container with id 7a37d664a8e38aa020896d08c7497f20a8ae17ebd87351c7a939f6d5f5bc6768 Jul 12 15:10:14.268641 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.268609 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:14.268741 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.268674 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:14.268741 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.268700 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:14.576821 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.576723 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbc51022-6147-481c-871b-556a23422245" path="/var/lib/kubelet/pods/bbc51022-6147-481c-871b-556a23422245/volumes" Jul 12 15:10:14.709917 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.709880 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" event={"ID":"d1d7ed85-7159-4f68-a543-881c9e5dbd64","Type":"ContainerStarted","Data":"d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81"} Jul 12 15:10:14.710089 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.709940 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" event={"ID":"d1d7ed85-7159-4f68-a543-881c9e5dbd64","Type":"ContainerStarted","Data":"7a37d664a8e38aa020896d08c7497f20a8ae17ebd87351c7a939f6d5f5bc6768"} Jul 12 15:10:14.733798 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.733749 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" podStartSLOduration=1.7337295190000002 podStartE2EDuration="1.733729519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:14.731835627 +0000 UTC m=+1762.699718881" watchObservedRunningTime="2026-07-12 15:10:14.733729519 +0000 UTC m=+1762.701612771" Jul 12 15:10:14.947505 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:14.947471 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:15.022102 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.022074 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:15.094048 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.094007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:15.134627 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.134600 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:15.143644 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.143620 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:15.195264 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.195229 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:15.514409 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.514381 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:15.615943 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.615913 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:15.714966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.714939 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:15.715876 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:15.715853 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:17.169383 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:17.169355 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:20.084130 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:20.084096 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:20.507675 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:20.507644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:25.767115 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:25.767086 2573 generic.go:356] "Generic (PLEG): container finished" podID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerID="d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81" exitCode=0 Jul 12 15:10:25.767523 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:25.767168 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" event={"ID":"d1d7ed85-7159-4f68-a543-881c9e5dbd64","Type":"ContainerDied","Data":"d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81"} Jul 12 15:10:25.767570 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:25.767539 2573 scope.go:117] "RemoveContainer" containerID="d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81" Jul 12 15:10:25.768541 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:25.768510 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:25.768769 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:25.768582 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:25.768769 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:25.768613 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:26.160657 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:26.160626 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp"] Jul 12 15:10:26.179407 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:26.179381 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:26.280615 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:26.280581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:26.773727 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:26.773696 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" event={"ID":"d1d7ed85-7159-4f68-a543-881c9e5dbd64","Type":"ContainerStarted","Data":"9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255"} Jul 12 15:10:26.774157 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:26.773818 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerName="istio-proxy" containerID="cri-o://9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255" gracePeriod=30 Jul 12 15:10:28.006599 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:28.006570 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:30.715145 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:30.715112 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:32.013023 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.012989 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:32.131869 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.131841 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-envoy\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132036 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.131876 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-token\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132036 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.131896 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istiod-ca-cert\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132036 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.131922 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-credential-socket\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132216 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132069 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcdxq\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-kube-api-access-vcdxq\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132216 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132114 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-socket\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132216 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132140 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-certs\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132216 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132166 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:32.132216 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132174 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-podinfo\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132216 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132211 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-data\") pod \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\" (UID: \"d1d7ed85-7159-4f68-a543-881c9e5dbd64\") " Jul 12 15:10:32.132497 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132327 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:32.132497 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132360 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:10:32.132497 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132373 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:32.132647 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132605 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.132647 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132625 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.132647 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132639 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.132803 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.132652 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.133242 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.133213 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:32.134099 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.134076 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:10:32.134196 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.134174 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-kube-api-access-vcdxq" (OuterVolumeSpecName: "kube-api-access-vcdxq") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "kube-api-access-vcdxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:10:32.134254 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.134217 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:32.134313 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.134289 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1d7ed85-7159-4f68-a543-881c9e5dbd64" (UID: "d1d7ed85-7159-4f68-a543-881c9e5dbd64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:10:32.139060 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.139001 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:32.196427 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.196401 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:32.233143 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.233122 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.233143 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.233142 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.233265 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.233151 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcdxq\" (UniqueName: \"kubernetes.io/projected/d1d7ed85-7159-4f68-a543-881c9e5dbd64-kube-api-access-vcdxq\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.233265 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.233160 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.233265 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.233169 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d7ed85-7159-4f68-a543-881c9e5dbd64-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:32.299966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.299934 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g"] Jul 12 15:10:32.300448 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.300424 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerName="istio-proxy" Jul 12 15:10:32.300448 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.300443 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerName="istio-proxy" Jul 12 15:10:32.300448 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.300452 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerName="istio-proxy" Jul 12 15:10:32.300617 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.300458 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerName="istio-proxy" Jul 12 15:10:32.300617 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.300534 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerName="istio-proxy" Jul 12 15:10:32.303197 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.303179 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.305187 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.305169 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9f3615f1-openshift-default-dockercfg-2qjgc\"" Jul 12 15:10:32.315256 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.315231 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g"] Jul 12 15:10:32.336888 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.336865 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:32.434987 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.434911 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-envoy\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.434987 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.434955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15b36569-99f9-4e21-9588-2395c1911a45-istiod-ca-cert\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.434987 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.434980 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6lsr\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-kube-api-access-k6lsr\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.435187 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.435046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-data\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.435187 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.435135 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15b36569-99f9-4e21-9588-2395c1911a45-istio-podinfo\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.435187 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.435157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-istio-token\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.435313 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.435193 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-certs\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.435313 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.435254 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-socket\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.435313 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.435280 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-credential-socket\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.436800 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.436782 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:32.536387 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-envoy\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536387 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536389 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15b36569-99f9-4e21-9588-2395c1911a45-istiod-ca-cert\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536550 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536408 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6lsr\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-kube-api-access-k6lsr\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536550 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536427 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-data\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536550 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15b36569-99f9-4e21-9588-2395c1911a45-istio-podinfo\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536550 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536492 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-istio-token\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536550 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536524 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-certs\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536795 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536560 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-socket\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536795 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536591 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-credential-socket\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536910 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536862 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-data\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.536989 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.536964 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-certs\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.537063 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.537006 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-credential-socket\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.537063 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.537053 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-socket\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.537230 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.537108 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15b36569-99f9-4e21-9588-2395c1911a45-istiod-ca-cert\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.538667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.538649 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-envoy\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.538895 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.538876 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15b36569-99f9-4e21-9588-2395c1911a45-istio-podinfo\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.543619 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.543594 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-istio-token\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.543700 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.543646 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6lsr\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-kube-api-access-k6lsr\") pod \"e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.614180 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.614155 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:32.739806 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.739779 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g"] Jul 12 15:10:32.741051 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:10:32.741027 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b36569_99f9_4e21_9588_2395c1911a45.slice/crio-a66bfcbed083bac8e74d309d455166ff054c454c5f05a029fd7f34b3151c1524 WatchSource:0}: Error finding container a66bfcbed083bac8e74d309d455166ff054c454c5f05a029fd7f34b3151c1524: Status 404 returned error can't find the container with id a66bfcbed083bac8e74d309d455166ff054c454c5f05a029fd7f34b3151c1524 Jul 12 15:10:32.742984 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.742949 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:32.743096 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.743060 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:32.743156 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.743112 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:10:32.805679 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.805650 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" event={"ID":"15b36569-99f9-4e21-9588-2395c1911a45","Type":"ContainerStarted","Data":"559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e"} Jul 12 15:10:32.805790 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.805707 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" event={"ID":"15b36569-99f9-4e21-9588-2395c1911a45","Type":"ContainerStarted","Data":"a66bfcbed083bac8e74d309d455166ff054c454c5f05a029fd7f34b3151c1524"} Jul 12 15:10:32.807429 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.807404 2573 generic.go:356] "Generic (PLEG): container finished" podID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerID="9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255" exitCode=0 Jul 12 15:10:32.807519 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.807438 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" Jul 12 15:10:32.807519 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.807454 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" event={"ID":"d1d7ed85-7159-4f68-a543-881c9e5dbd64","Type":"ContainerDied","Data":"9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255"} Jul 12 15:10:32.807519 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.807502 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp" event={"ID":"d1d7ed85-7159-4f68-a543-881c9e5dbd64","Type":"ContainerDied","Data":"7a37d664a8e38aa020896d08c7497f20a8ae17ebd87351c7a939f6d5f5bc6768"} Jul 12 15:10:32.807627 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.807532 2573 scope.go:117] "RemoveContainer" containerID="9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255" Jul 12 15:10:32.815961 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.815939 2573 scope.go:117] "RemoveContainer" containerID="d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81" Jul 12 15:10:32.824795 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.824772 2573 scope.go:117] "RemoveContainer" containerID="9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255" Jul 12 15:10:32.825160 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:10:32.825133 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255\": container with ID starting with 9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255 not found: ID does not exist" containerID="9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255" Jul 12 15:10:32.825279 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.825170 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255"} err="failed to get container status \"9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255\": rpc error: code = NotFound desc = could not find container \"9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255\": container with ID starting with 9bf8a772a7aa31fa5878625aa75847953d27c289f0ab677aca9673eac6e9c255 not found: ID does not exist" Jul 12 15:10:32.825279 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.825193 2573 scope.go:117] "RemoveContainer" containerID="d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81" Jul 12 15:10:32.825416 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:10:32.825394 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81\": container with ID starting with d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81 not found: ID does not exist" containerID="d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81" Jul 12 15:10:32.825512 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.825421 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81"} err="failed to get container status \"d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81\": rpc error: code = NotFound desc = could not find container \"d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81\": container with ID starting with d9326b1331172be22c5897b895b5e55fcae477c583ffd134e96de6a158154f81 not found: ID does not exist" Jul 12 15:10:32.825695 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.825657 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" podStartSLOduration=0.825643809 podStartE2EDuration="825.643809ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:10:32.821664745 +0000 UTC m=+1780.789547999" watchObservedRunningTime="2026-07-12 15:10:32.825643809 +0000 UTC m=+1780.793527061" Jul 12 15:10:32.834111 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.834088 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp"] Jul 12 15:10:32.837307 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:32.837287 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-61e6b661-openshift-default-67ff7cfbd5-6nwlp"] Jul 12 15:10:33.433585 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.433553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:33.511837 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.511800 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:33.582776 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.582745 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:33.614682 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.614650 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:33.622908 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.622884 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:33.814639 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.814559 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:33.815374 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.815356 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:33.994366 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:33.994334 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:34.095879 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:34.095811 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:34.195956 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:34.195924 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:34.575702 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:34.575669 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" path="/var/lib/kubelet/pods/d1d7ed85-7159-4f68-a543-881c9e5dbd64/volumes" Jul 12 15:10:35.451694 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:35.451666 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:37.887728 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:37.887698 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:38.040328 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:38.040291 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g"] Jul 12 15:10:38.040520 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:38.040497 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" podUID="15b36569-99f9-4e21-9588-2395c1911a45" containerName="istio-proxy" containerID="cri-o://559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e" gracePeriod=30 Jul 12 15:10:38.067417 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:38.067376 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:38.166075 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:38.165973 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:38.239346 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:38.239313 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:39.410218 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:39.410181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:43.593751 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.593728 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:43.727391 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727298 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-envoy\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727391 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727340 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6lsr\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-kube-api-access-k6lsr\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727391 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727364 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-certs\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727394 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15b36569-99f9-4e21-9588-2395c1911a45-istio-podinfo\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727428 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-istio-token\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727447 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-credential-socket\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727503 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15b36569-99f9-4e21-9588-2395c1911a45-istiod-ca-cert\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727570 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-socket\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727609 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-data\") pod \"15b36569-99f9-4e21-9588-2395c1911a45\" (UID: \"15b36569-99f9-4e21-9588-2395c1911a45\") " Jul 12 15:10:43.727967 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727683 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:43.727967 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727777 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:43.727967 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727956 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.728158 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.727976 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.728158 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.728073 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b36569-99f9-4e21-9588-2395c1911a45-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:10:43.728287 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.728199 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:43.728466 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.728446 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-data" (OuterVolumeSpecName: "istio-data") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:43.729560 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.729536 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15b36569-99f9-4e21-9588-2395c1911a45-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:10:43.729678 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.729657 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-kube-api-access-k6lsr" (OuterVolumeSpecName: "kube-api-access-k6lsr") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "kube-api-access-k6lsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:10:43.729943 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.729918 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-istio-token" (OuterVolumeSpecName: "istio-token") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:10:43.729943 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.729920 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15b36569-99f9-4e21-9588-2395c1911a45" (UID: "15b36569-99f9-4e21-9588-2395c1911a45"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:10:43.792181 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.792156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:43.828685 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.828663 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.828685 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.828683 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15b36569-99f9-4e21-9588-2395c1911a45-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.828863 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.828692 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.828863 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.828700 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.828863 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.828709 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15b36569-99f9-4e21-9588-2395c1911a45-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.828863 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.828717 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6lsr\" (UniqueName: \"kubernetes.io/projected/15b36569-99f9-4e21-9588-2395c1911a45-kube-api-access-k6lsr\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.828863 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.828726 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15b36569-99f9-4e21-9588-2395c1911a45-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:10:43.856411 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.856383 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:43.864216 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.864195 2573 generic.go:356] "Generic (PLEG): container finished" podID="15b36569-99f9-4e21-9588-2395c1911a45" containerID="559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e" exitCode=0 Jul 12 15:10:43.864323 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.864226 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" event={"ID":"15b36569-99f9-4e21-9588-2395c1911a45","Type":"ContainerDied","Data":"559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e"} Jul 12 15:10:43.864323 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.864284 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" event={"ID":"15b36569-99f9-4e21-9588-2395c1911a45","Type":"ContainerDied","Data":"a66bfcbed083bac8e74d309d455166ff054c454c5f05a029fd7f34b3151c1524"} Jul 12 15:10:43.864323 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.864311 2573 scope.go:117] "RemoveContainer" containerID="559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e" Jul 12 15:10:43.864323 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.864238 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g" Jul 12 15:10:43.873630 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.873614 2573 scope.go:117] "RemoveContainer" containerID="559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e" Jul 12 15:10:43.873859 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:10:43.873834 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e\": container with ID starting with 559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e not found: ID does not exist" containerID="559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e" Jul 12 15:10:43.873919 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.873866 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e"} err="failed to get container status \"559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e\": rpc error: code = NotFound desc = could not find container \"559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e\": container with ID starting with 559c504ac8c7264051fee56d7fa5caa4bfbef0adc9d3cdf17f1792bf4f21bc5e not found: ID does not exist" Jul 12 15:10:43.884891 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.884872 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g"] Jul 12 15:10:43.889248 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:43.889228 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9f3615f1-openshift-default-5769578ccf-z485g"] Jul 12 15:10:44.232301 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:44.232267 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:44.328371 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:44.328337 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:44.428340 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:44.428306 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:44.576139 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:44.576050 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15b36569-99f9-4e21-9588-2395c1911a45" path="/var/lib/kubelet/pods/15b36569-99f9-4e21-9588-2395c1911a45/volumes" Jul 12 15:10:45.756719 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:45.756685 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:51.698003 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:51.697966 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:51.798302 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:51.798268 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:51.898161 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:51.898131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:51.997880 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:51.997798 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:52.378282 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:52.378230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:10:52.623667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:52.623637 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:10:52.630746 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:52.630669 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:10:52.631123 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:52.631104 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:10:52.637300 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:10:52.637280 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:11:18.315655 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:18.315622 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:24.085434 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:24.085398 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:24.142374 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:24.142342 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:24.209239 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:24.209189 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:24.309469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:24.309435 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:40.146752 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:40.146720 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:45.911682 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:45.911644 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:46.020974 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:46.020929 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:46.128082 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:46.128048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:56.634935 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:56.634900 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:11:57.412507 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:11:57.412472 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:03.091820 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:03.091782 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:03.145193 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:03.145166 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:03.256557 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:03.256526 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:03.352202 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:03.352134 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:03.451537 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:03.451509 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:04.176385 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:04.176338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:04.274152 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:04.274110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:04.378457 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:04.378431 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:05.927753 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:05.927706 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:11.891638 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:11.891598 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:11.949852 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:11.949827 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:12.575244 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:12.575209 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:18.329989 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.329898 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:18.429961 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.429923 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:18.562624 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.562594 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:18.840072 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.840035 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz"] Jul 12 15:12:18.840413 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.840398 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15b36569-99f9-4e21-9588-2395c1911a45" containerName="istio-proxy" Jul 12 15:12:18.840490 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.840413 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="15b36569-99f9-4e21-9588-2395c1911a45" containerName="istio-proxy" Jul 12 15:12:18.840549 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.840502 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="15b36569-99f9-4e21-9588-2395c1911a45" containerName="istio-proxy" Jul 12 15:12:18.840549 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.840516 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d7ed85-7159-4f68-a543-881c9e5dbd64" containerName="istio-proxy" Jul 12 15:12:18.843536 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.843517 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.846146 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.846127 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ea99faec-openshift-default-dockercfg-xqr9b\"" Jul 12 15:12:18.864700 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.864676 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz"] Jul 12 15:12:18.877880 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.877859 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:18.890667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.890643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-certs\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.890765 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.890684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-token\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.890765 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.890716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-credential-socket\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.890852 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.890809 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-envoy\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.890852 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.890845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-data\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.890951 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.890916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-podinfo\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.891009 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.890993 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8bs7\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-kube-api-access-h8bs7\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.891101 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.891057 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istiod-ca-cert\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.891155 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.891103 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-socket\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991514 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-envoy\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991514 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-data\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991718 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991544 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-podinfo\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991718 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991592 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8bs7\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-kube-api-access-h8bs7\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991718 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991622 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istiod-ca-cert\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991881 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-socket\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991881 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-certs\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.991881 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991846 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-token\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.992075 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991896 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-credential-socket\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.992075 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.991962 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-data\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.992190 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.992111 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-socket\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.992247 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.992188 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-certs\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.992307 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.992275 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-credential-socket\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.992641 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.992613 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istiod-ca-cert\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.993757 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.993735 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-envoy\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.994006 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.993988 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-podinfo\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.998839 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.998818 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-token\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:18.999035 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:18.998999 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8bs7\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-kube-api-access-h8bs7\") pod \"e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:19.000129 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:19.000096 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:19.154561 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:19.154529 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:19.288190 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:19.288161 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz"] Jul 12 15:12:19.289866 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:12:19.289843 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa17b06_e484_4fd2_a9d3_3daf87ad8a0d.slice/crio-38a6ec88ae0c19ff0d5197e8113f84d572db9a48f4c0e70a2eff6c43d1bed430 WatchSource:0}: Error finding container 38a6ec88ae0c19ff0d5197e8113f84d572db9a48f4c0e70a2eff6c43d1bed430: Status 404 returned error can't find the container with id 38a6ec88ae0c19ff0d5197e8113f84d572db9a48f4c0e70a2eff6c43d1bed430 Jul 12 15:12:19.292278 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:19.292245 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:12:19.292352 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:19.292318 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:12:19.292352 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:19.292345 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:12:19.334282 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:19.334254 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" event={"ID":"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d","Type":"ContainerStarted","Data":"38a6ec88ae0c19ff0d5197e8113f84d572db9a48f4c0e70a2eff6c43d1bed430"} Jul 12 15:12:20.340926 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:20.340891 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" event={"ID":"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d","Type":"ContainerStarted","Data":"b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07"} Jul 12 15:12:20.360240 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:20.360187 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" podStartSLOduration=2.360172822 podStartE2EDuration="2.360172822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:12:20.358649383 +0000 UTC m=+1888.326532638" watchObservedRunningTime="2026-07-12 15:12:20.360172822 +0000 UTC m=+1888.328056075" Jul 12 15:12:21.155177 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:21.155140 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:21.159515 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:21.159491 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:21.346115 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:21.346084 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:21.347039 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:21.346996 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:22.261597 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:22.261563 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:22.340785 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:22.340741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:22.417705 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:22.417670 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:22.570096 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:22.569986 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:22.932387 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:22.932360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:23.032994 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:23.032962 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:26.748847 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:26.748800 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:27.389174 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:27.389136 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:27.491322 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:27.491284 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:27.824956 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:27.824861 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:33.556740 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:33.556687 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:33.617254 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:33.617225 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:34.094071 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.094031 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:34.415981 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.415947 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerID="b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07" exitCode=0 Jul 12 15:12:34.416171 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.416043 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" event={"ID":"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d","Type":"ContainerDied","Data":"b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07"} Jul 12 15:12:34.416426 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.416412 2573 scope.go:117] "RemoveContainer" containerID="b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07" Jul 12 15:12:34.417677 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.417533 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:12:34.417677 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.417622 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:12:34.417677 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.417672 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:12:34.443548 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.443501 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:34.536186 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:34.536150 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:35.422721 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:35.422689 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" event={"ID":"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d","Type":"ContainerStarted","Data":"c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690"} Jul 12 15:12:36.155174 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:36.155137 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:36.156323 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:36.156287 2573 patch_prober.go:28] interesting pod/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 12 15:12:36.156468 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:36.156348 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 12 15:12:36.346546 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:36.346513 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:37.155557 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:37.155515 2573 patch_prober.go:28] interesting pod/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 12 15:12:37.156051 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:37.155578 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 12 15:12:38.155823 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:38.155787 2573 patch_prober.go:28] interesting pod/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 12 15:12:38.156424 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:38.156392 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 12 15:12:39.171475 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:39.171444 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:39.172685 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:39.172665 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:40.279306 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:40.279270 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:40.346166 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:40.346132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:40.509291 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:40.509255 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz"] Jul 12 15:12:40.509514 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:40.509487 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" containerID="cri-o://c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690" gracePeriod=30 Jul 12 15:12:40.552771 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:40.552706 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:40.859409 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:40.859321 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:40.958477 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:40.958438 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:41.073242 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:41.073208 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:45.759063 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.759027 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:45.940261 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940225 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-socket\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940451 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940274 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istiod-ca-cert\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940451 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940311 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-credential-socket\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940451 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940353 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-token\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940451 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940379 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-envoy\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940451 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940420 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-data\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940472 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-certs\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940496 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8bs7\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-kube-api-access-h8bs7\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940568 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:12:45.940715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940574 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-podinfo\") pod \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\" (UID: \"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d\") " Jul 12 15:12:45.940715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940682 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:12:45.940961 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940908 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:45.940961 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940929 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:45.940961 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940905 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:12:45.940961 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.940902 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:12:45.941534 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.941505 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:12:45.942703 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.942676 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:12:45.942703 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.942668 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-kube-api-access-h8bs7" (OuterVolumeSpecName: "kube-api-access-h8bs7") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "kube-api-access-h8bs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:12:45.942703 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.942672 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:12:45.942852 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:45.942703 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" (UID: "3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:12:46.041740 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.041709 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:46.041740 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.041742 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8bs7\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-kube-api-access-h8bs7\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:46.041936 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.041757 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:46.041936 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.041771 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:46.041936 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.041779 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:46.041936 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.041787 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:46.041936 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.041795 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:12:46.478673 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.478637 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerID="c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690" exitCode=0 Jul 12 15:12:46.478865 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.478690 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" Jul 12 15:12:46.478865 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.478714 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" event={"ID":"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d","Type":"ContainerDied","Data":"c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690"} Jul 12 15:12:46.478865 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.478767 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz" event={"ID":"3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d","Type":"ContainerDied","Data":"38a6ec88ae0c19ff0d5197e8113f84d572db9a48f4c0e70a2eff6c43d1bed430"} Jul 12 15:12:46.478865 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.478789 2573 scope.go:117] "RemoveContainer" containerID="c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690" Jul 12 15:12:46.487817 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.487800 2573 scope.go:117] "RemoveContainer" containerID="b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07" Jul 12 15:12:46.495309 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.495292 2573 scope.go:117] "RemoveContainer" containerID="c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690" Jul 12 15:12:46.495531 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:12:46.495505 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690\": container with ID starting with c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690 not found: ID does not exist" containerID="c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690" Jul 12 15:12:46.495577 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.495533 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690"} err="failed to get container status \"c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690\": rpc error: code = NotFound desc = could not find container \"c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690\": container with ID starting with c4911093199d3be370bafe1d7c6f9afbcffbc08629235280fc913f7189359690 not found: ID does not exist" Jul 12 15:12:46.495577 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.495555 2573 scope.go:117] "RemoveContainer" containerID="b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07" Jul 12 15:12:46.495771 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:12:46.495754 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07\": container with ID starting with b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07 not found: ID does not exist" containerID="b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07" Jul 12 15:12:46.495803 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.495778 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07"} err="failed to get container status \"b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07\": rpc error: code = NotFound desc = could not find container \"b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07\": container with ID starting with b58baaf93ea00ddb9ded1e45453ddefc3cbdefcfb1c07c29bae0f895dbd22a07 not found: ID does not exist" Jul 12 15:12:46.499237 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.499217 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz"] Jul 12 15:12:46.502937 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.502917 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ea99faec-openshift-default-bbbd75f75-w75nz"] Jul 12 15:12:46.576370 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.576347 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" path="/var/lib/kubelet/pods/3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d/volumes" Jul 12 15:12:46.876324 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.876298 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:46.935783 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:46.935756 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:47.016652 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:47.016619 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:47.114533 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:47.114502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:47.213737 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:47.213661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:52.755999 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:52.755967 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:52.857156 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:52.857121 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:59.588539 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.588504 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:59.869473 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.869384 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc"] Jul 12 15:12:59.869850 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.869831 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" Jul 12 15:12:59.869930 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.869852 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" Jul 12 15:12:59.869930 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.869924 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" Jul 12 15:12:59.870007 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.869934 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" Jul 12 15:12:59.872989 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.872962 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.875003 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.874982 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-90571a9f-openshift-default-dockercfg-v8tpv\"" Jul 12 15:12:59.884120 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.884097 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc"] Jul 12 15:12:59.896768 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.896740 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:12:59.961400 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961359 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istiod-ca-cert\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961569 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brvjg\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-kube-api-access-brvjg\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961569 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-data\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961569 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-podinfo\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961569 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961554 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-token\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961704 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-socket\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961704 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-credential-socket\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961769 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961746 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-envoy\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:12:59.961805 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:12:59.961776 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-certs\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.045812 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.045778 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:00.062556 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062527 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brvjg\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-kube-api-access-brvjg\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062683 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062564 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-data\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062683 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062583 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-podinfo\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062683 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062605 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-token\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062683 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062659 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-socket\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062859 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-credential-socket\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062859 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062751 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-envoy\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062859 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-certs\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.062859 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.062829 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istiod-ca-cert\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.063160 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.063133 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-credential-socket\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.063240 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.063186 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-data\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.063240 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.063202 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-socket\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.063328 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.063248 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-certs\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.063514 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.063496 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istiod-ca-cert\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.064955 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.064934 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-envoy\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.065171 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.065155 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-podinfo\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.069926 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.069897 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-token\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.070065 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.070047 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brvjg\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-kube-api-access-brvjg\") pod \"e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.184059 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.183982 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:00.319160 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.319133 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc"] Jul 12 15:13:00.320903 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:13:00.320876 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a2f58d_fcc4_4839_a6f8_d0ec21891a2b.slice/crio-91c93156c59f61b3322f2fc0d399bbce30b9094564232b295ae8c8175cc68607 WatchSource:0}: Error finding container 91c93156c59f61b3322f2fc0d399bbce30b9094564232b295ae8c8175cc68607: Status 404 returned error can't find the container with id 91c93156c59f61b3322f2fc0d399bbce30b9094564232b295ae8c8175cc68607 Jul 12 15:13:00.323132 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.323101 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:13:00.323221 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.323164 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:13:00.323221 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.323194 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:13:00.549579 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.549469 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" event={"ID":"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b","Type":"ContainerStarted","Data":"0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87"} Jul 12 15:13:00.549579 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.549544 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" event={"ID":"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b","Type":"ContainerStarted","Data":"91c93156c59f61b3322f2fc0d399bbce30b9094564232b295ae8c8175cc68607"} Jul 12 15:13:00.568944 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:00.568886 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" podStartSLOduration=1.568870965 podStartE2EDuration="1.568870965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:13:00.566009298 +0000 UTC m=+1928.533892550" watchObservedRunningTime="2026-07-12 15:13:00.568870965 +0000 UTC m=+1928.536754209" Jul 12 15:13:01.184297 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:01.184253 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:01.189166 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:01.189141 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:01.554982 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:01.554901 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:01.555880 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:01.555859 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:02.449272 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:02.449240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:02.558967 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:02.558931 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:02.671171 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:02.671132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:02.771438 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:02.771344 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:03.213887 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:03.213851 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:03.315855 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:03.315816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:06.979559 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:06.979525 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:12.988980 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.988891 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4"] Jul 12 15:13:12.989494 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.989236 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" Jul 12 15:13:12.989494 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.989247 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa17b06-e484-4fd2-a9d3-3daf87ad8a0d" containerName="istio-proxy" Jul 12 15:13:12.992428 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.992408 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:12.994935 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.994911 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-90571a9f\"/\"kube-root-ca.crt\"" Jul 12 15:13:12.995122 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.994914 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-90571a9f\"/\"default-dockercfg-tfp4v\"" Jul 12 15:13:12.995324 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.995306 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-90571a9f\"/\"openshift-service-ca.crt\"" Jul 12 15:13:12.995386 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:12.995366 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-90571a9f\"/\"e2e-lifecycle-model-90571a9f-kserve-self-signed-certs\"" Jul 12 15:13:13.004131 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.004104 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4"] Jul 12 15:13:13.072517 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.072487 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:13.081293 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.081267 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-dshm\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.081394 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.081314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-model-cache\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.081394 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.081340 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-tmp-dir\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.081470 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.081400 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl5pr\" (UniqueName: \"kubernetes.io/projected/5a8941a2-d163-4903-89e6-973938eeb50f-kube-api-access-kl5pr\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.081526 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.081505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-home\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.081615 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.081546 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8941a2-d163-4903-89e6-973938eeb50f-tls-certs\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.182746 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.182697 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-dshm\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.182952 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.182795 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-model-cache\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.182952 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.182820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-tmp-dir\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.182952 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.182863 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl5pr\" (UniqueName: \"kubernetes.io/projected/5a8941a2-d163-4903-89e6-973938eeb50f-kube-api-access-kl5pr\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.182952 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.182940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-home\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.183212 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.182962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8941a2-d163-4903-89e6-973938eeb50f-tls-certs\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.183330 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.183269 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-model-cache\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.183473 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.183391 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-tmp-dir\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.183473 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.183438 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-home\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.185304 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.185274 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-dshm\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.185677 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.185655 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8941a2-d163-4903-89e6-973938eeb50f-tls-certs\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.193390 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.193361 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl5pr\" (UniqueName: \"kubernetes.io/projected/5a8941a2-d163-4903-89e6-973938eeb50f-kube-api-access-kl5pr\") pod \"e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.301946 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.301844 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:13.443952 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.443922 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4"] Jul 12 15:13:13.446932 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:13:13.446895 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a8941a2_d163_4903_89e6_973938eeb50f.slice/crio-bb58ee7f8bfb381af63eaa958a162d42353f52473db00bd0afc8a29931afaaa5 WatchSource:0}: Error finding container bb58ee7f8bfb381af63eaa958a162d42353f52473db00bd0afc8a29931afaaa5: Status 404 returned error can't find the container with id bb58ee7f8bfb381af63eaa958a162d42353f52473db00bd0afc8a29931afaaa5 Jul 12 15:13:13.615416 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.615314 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" event={"ID":"5a8941a2-d163-4903-89e6-973938eeb50f","Type":"ContainerStarted","Data":"d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d"} Jul 12 15:13:13.615416 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.615361 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" event={"ID":"5a8941a2-d163-4903-89e6-973938eeb50f","Type":"ContainerStarted","Data":"bb58ee7f8bfb381af63eaa958a162d42353f52473db00bd0afc8a29931afaaa5"} Jul 12 15:13:13.630758 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:13.630666 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" podStartSLOduration=1.6306467260000002 podStartE2EDuration="1.630646726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:13:13.630587006 +0000 UTC m=+1941.598470259" watchObservedRunningTime="2026-07-12 15:13:13.630646726 +0000 UTC m=+1941.598529978" Jul 12 15:13:15.627949 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:15.627919 2573 generic.go:356] "Generic (PLEG): container finished" podID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerID="0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87" exitCode=0 Jul 12 15:13:15.628420 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:15.627990 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" event={"ID":"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b","Type":"ContainerDied","Data":"0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87"} Jul 12 15:13:15.628420 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:15.628410 2573 scope.go:117] "RemoveContainer" containerID="0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87" Jul 12 15:13:15.629419 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:15.629388 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:13:15.629491 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:15.629460 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:13:15.629529 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:15.629494 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:13:16.555380 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:16.555326 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:16.635103 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:16.635059 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" event={"ID":"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b","Type":"ContainerStarted","Data":"a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8"} Jul 12 15:13:17.184791 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:17.184752 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:17.185793 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:17.185766 2573 patch_prober.go:28] interesting pod/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 15:13:17.185926 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:17.185823 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 15:13:18.184623 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:18.184580 2573 patch_prober.go:28] interesting pod/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 15:13:18.185062 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:18.184666 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 15:13:19.184649 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:19.184609 2573 patch_prober.go:28] interesting pod/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 15:13:19.185067 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:19.184681 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 15:13:20.185328 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:20.185288 2573 patch_prober.go:28] interesting pod/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 15:13:20.185734 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:20.185348 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 15:13:21.188848 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:21.188817 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:21.189266 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:21.189204 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:21.189976 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:21.189954 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:13:23.302183 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:23.302146 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:23.302674 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:23.302198 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:23.314038 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:23.313999 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:23.681594 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:23.681565 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:13:25.524684 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:25.524651 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:34.241797 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:34.241763 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:34.716248 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:34.716195 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:37.724844 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:37.724803 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:37.838121 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:37.838091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:40.343005 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:40.342960 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:13:45.314625 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:45.314546 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4"] Jul 12 15:13:45.354910 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:13:45.354881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:00.334483 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.334438 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" podUID="5a8941a2-d163-4903-89e6-973938eeb50f" containerName="main" containerID="cri-o://d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d" gracePeriod=45 Jul 12 15:14:00.579737 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.579714 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:14:00.664603 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.664573 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-tmp-dir\") pod \"5a8941a2-d163-4903-89e6-973938eeb50f\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " Jul 12 15:14:00.664753 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.664655 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl5pr\" (UniqueName: \"kubernetes.io/projected/5a8941a2-d163-4903-89e6-973938eeb50f-kube-api-access-kl5pr\") pod \"5a8941a2-d163-4903-89e6-973938eeb50f\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " Jul 12 15:14:00.664753 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.664701 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-dshm\") pod \"5a8941a2-d163-4903-89e6-973938eeb50f\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " Jul 12 15:14:00.664753 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.664734 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-home\") pod \"5a8941a2-d163-4903-89e6-973938eeb50f\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " Jul 12 15:14:00.664914 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.664801 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8941a2-d163-4903-89e6-973938eeb50f-tls-certs\") pod \"5a8941a2-d163-4903-89e6-973938eeb50f\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " Jul 12 15:14:00.664914 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.664857 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-model-cache\") pod \"5a8941a2-d163-4903-89e6-973938eeb50f\" (UID: \"5a8941a2-d163-4903-89e6-973938eeb50f\") " Jul 12 15:14:00.665034 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.664855 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5a8941a2-d163-4903-89e6-973938eeb50f" (UID: "5a8941a2-d163-4903-89e6-973938eeb50f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:00.665091 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.665038 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-home" (OuterVolumeSpecName: "home") pod "5a8941a2-d163-4903-89e6-973938eeb50f" (UID: "5a8941a2-d163-4903-89e6-973938eeb50f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:00.665160 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.665143 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-model-cache" (OuterVolumeSpecName: "model-cache") pod "5a8941a2-d163-4903-89e6-973938eeb50f" (UID: "5a8941a2-d163-4903-89e6-973938eeb50f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:00.665328 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.665306 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-model-cache\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:00.665399 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.665335 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-tmp-dir\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:00.665399 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.665349 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-home\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:00.667263 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.667227 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-dshm" (OuterVolumeSpecName: "dshm") pod "5a8941a2-d163-4903-89e6-973938eeb50f" (UID: "5a8941a2-d163-4903-89e6-973938eeb50f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:00.672863 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.672832 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a8941a2-d163-4903-89e6-973938eeb50f-kube-api-access-kl5pr" (OuterVolumeSpecName: "kube-api-access-kl5pr") pod "5a8941a2-d163-4903-89e6-973938eeb50f" (UID: "5a8941a2-d163-4903-89e6-973938eeb50f"). InnerVolumeSpecName "kube-api-access-kl5pr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:14:00.672956 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.672908 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a8941a2-d163-4903-89e6-973938eeb50f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5a8941a2-d163-4903-89e6-973938eeb50f" (UID: "5a8941a2-d163-4903-89e6-973938eeb50f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:14:00.766754 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.766727 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8941a2-d163-4903-89e6-973938eeb50f-dshm\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:00.766754 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.766751 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8941a2-d163-4903-89e6-973938eeb50f-tls-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:00.766914 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.766778 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl5pr\" (UniqueName: \"kubernetes.io/projected/5a8941a2-d163-4903-89e6-973938eeb50f-kube-api-access-kl5pr\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:00.857483 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.857454 2573 generic.go:356] "Generic (PLEG): container finished" podID="5a8941a2-d163-4903-89e6-973938eeb50f" containerID="d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d" exitCode=0 Jul 12 15:14:00.857611 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.857505 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" Jul 12 15:14:00.857611 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.857503 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" event={"ID":"5a8941a2-d163-4903-89e6-973938eeb50f","Type":"ContainerDied","Data":"d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d"} Jul 12 15:14:00.857611 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.857558 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4" event={"ID":"5a8941a2-d163-4903-89e6-973938eeb50f","Type":"ContainerDied","Data":"bb58ee7f8bfb381af63eaa958a162d42353f52473db00bd0afc8a29931afaaa5"} Jul 12 15:14:00.857611 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.857582 2573 scope.go:117] "RemoveContainer" containerID="d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d" Jul 12 15:14:00.866471 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.866454 2573 scope.go:117] "RemoveContainer" containerID="d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d" Jul 12 15:14:00.866735 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:14:00.866716 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d\": container with ID starting with d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d not found: ID does not exist" containerID="d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d" Jul 12 15:14:00.866806 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.866743 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d"} err="failed to get container status \"d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d\": rpc error: code = NotFound desc = could not find container \"d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d\": container with ID starting with d10641859e152b6209a84094c773fbe7c005e2f55082f9e4b1c4087f95caeb1d not found: ID does not exist" Jul 12 15:14:00.878305 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.878286 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4"] Jul 12 15:14:00.886441 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:00.886422 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-90571a9f/e2e-lifecycle-model-90571a9f-kserve-588c6778dc-sczf4"] Jul 12 15:14:02.576139 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:02.576102 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8941a2-d163-4903-89e6-973938eeb50f" path="/var/lib/kubelet/pods/5a8941a2-d163-4903-89e6-973938eeb50f/volumes" Jul 12 15:14:22.409424 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:22.409391 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:22.512711 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:22.512683 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:28.026102 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:28.026064 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:28.127869 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:28.127835 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:28.346610 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:28.346532 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc"] Jul 12 15:14:28.346785 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:28.346743 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" containerID="cri-o://a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8" gracePeriod=30 Jul 12 15:14:28.375715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:28.375689 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:29.620908 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:29.620873 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:29.716153 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:29.716125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:29.815752 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:29.815726 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:30.617536 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:30.617507 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:30.713051 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:30.712999 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:30.813145 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:30.813114 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:32.253163 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:32.253128 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:33.596771 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.596749 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:14:33.730444 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730404 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brvjg\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-kube-api-access-brvjg\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730444 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730449 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-certs\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730679 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730478 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-token\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730679 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730499 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-data\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730679 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730515 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-podinfo\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730679 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730544 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-socket\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730679 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730626 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istiod-ca-cert\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730934 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730682 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-credential-socket\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730934 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730702 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-envoy\") pod \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\" (UID: \"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b\") " Jul 12 15:14:33.730934 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730847 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:33.730934 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.730866 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:33.731201 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.731048 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.731201 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.731072 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.731201 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.731190 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:14:33.731362 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.731226 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:33.731662 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.731632 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:33.732704 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.732672 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:14:33.732800 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.732677 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:14:33.732800 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.732750 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-kube-api-access-brvjg" (OuterVolumeSpecName: "kube-api-access-brvjg") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "kube-api-access-brvjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:14:33.732877 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.732823 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" (UID: "a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:14:33.832105 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.832075 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.832105 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.832106 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.832297 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.832119 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.832297 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.832130 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.832297 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.832142 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.832297 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.832155 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brvjg\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-kube-api-access-brvjg\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:33.832297 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:33.832167 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:14:34.022708 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.022617 2573 generic.go:356] "Generic (PLEG): container finished" podID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerID="a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8" exitCode=0 Jul 12 15:14:34.022708 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.022683 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" Jul 12 15:14:34.022912 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.022702 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" event={"ID":"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b","Type":"ContainerDied","Data":"a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8"} Jul 12 15:14:34.022912 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.022781 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc" event={"ID":"a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b","Type":"ContainerDied","Data":"91c93156c59f61b3322f2fc0d399bbce30b9094564232b295ae8c8175cc68607"} Jul 12 15:14:34.022912 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.022805 2573 scope.go:117] "RemoveContainer" containerID="a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8" Jul 12 15:14:34.032258 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.032243 2573 scope.go:117] "RemoveContainer" containerID="0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87" Jul 12 15:14:34.040379 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.040362 2573 scope.go:117] "RemoveContainer" containerID="a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8" Jul 12 15:14:34.040636 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:14:34.040619 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8\": container with ID starting with a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8 not found: ID does not exist" containerID="a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8" Jul 12 15:14:34.040672 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.040645 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8"} err="failed to get container status \"a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8\": rpc error: code = NotFound desc = could not find container \"a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8\": container with ID starting with a4c8e5393539d8a5148a17cb98a79353ad8d3d8f4ae11f8a1b68218ca20cfcc8 not found: ID does not exist" Jul 12 15:14:34.040672 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.040661 2573 scope.go:117] "RemoveContainer" containerID="0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87" Jul 12 15:14:34.040876 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:14:34.040858 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87\": container with ID starting with 0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87 not found: ID does not exist" containerID="0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87" Jul 12 15:14:34.040917 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.040883 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87"} err="failed to get container status \"0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87\": rpc error: code = NotFound desc = could not find container \"0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87\": container with ID starting with 0686e8b78fa124d4163d7b11f5cb4e22f8adbca37d8fa6ea017d6e4c47537b87 not found: ID does not exist" Jul 12 15:14:34.043676 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.043656 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc"] Jul 12 15:14:34.047611 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.047593 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-90571a9f-openshift-default-7f4bcd6f87-8rvdc"] Jul 12 15:14:34.575737 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:34.575704 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" path="/var/lib/kubelet/pods/a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b/volumes" Jul 12 15:14:38.080807 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:38.080768 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:38.153951 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:38.153912 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:38.739167 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:38.739133 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:44.684263 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:44.684227 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:44.788662 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:44.788631 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:44.885963 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:44.885933 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:55.369090 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:55.369056 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:14:55.719314 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:14:55.719285 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:15.148583 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:15.148506 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:16.695529 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:16.695496 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:22.341143 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.341106 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:22.441004 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.440970 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:22.675195 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.675167 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:22.943501 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943423 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm"] Jul 12 15:15:22.943899 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943882 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" Jul 12 15:15:22.943965 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943901 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" Jul 12 15:15:22.943965 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943911 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" Jul 12 15:15:22.943965 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943917 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" Jul 12 15:15:22.943965 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943924 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8941a2-d163-4903-89e6-973938eeb50f" containerName="main" Jul 12 15:15:22.943965 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943930 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8941a2-d163-4903-89e6-973938eeb50f" containerName="main" Jul 12 15:15:22.944138 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943988 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8941a2-d163-4903-89e6-973938eeb50f" containerName="main" Jul 12 15:15:22.944138 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.943995 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" Jul 12 15:15:22.944138 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.944001 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8a2f58d-fcc4-4839-a6f8-d0ec21891a2b" containerName="istio-proxy" Jul 12 15:15:22.947026 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.946993 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.948978 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.948957 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-43c370-openshift-default-dockercfg-mdz7q\"" Jul 12 15:15:22.972281 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.972258 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm"] Jul 12 15:15:22.978729 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.978707 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:22.999328 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999305 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-748wr\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-kube-api-access-748wr\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999424 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999335 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-podinfo\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999424 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999357 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-socket\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999424 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999379 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-credential-socket\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999525 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istiod-ca-cert\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999525 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-envoy\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999599 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999535 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-certs\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999599 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999570 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-data\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:22.999599 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:22.999588 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-token\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100209 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100174 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-token\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100355 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-748wr\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-kube-api-access-748wr\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100355 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100262 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-podinfo\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100355 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100296 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-socket\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100355 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-credential-socket\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100575 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100391 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istiod-ca-cert\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100575 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100434 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-envoy\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100575 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100482 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-certs\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100575 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-data\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100854 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100833 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-credential-socket\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100925 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100883 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-certs\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.100967 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.100938 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-data\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.101062 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.101026 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-socket\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.101208 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.101187 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istiod-ca-cert\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.102720 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.102697 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-envoy\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.102913 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.102897 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-podinfo\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.107820 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.107790 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-748wr\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-kube-api-access-748wr\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.108167 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.108149 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-token\") pod \"e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.257359 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.257291 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:23.392577 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.392546 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm"] Jul 12 15:15:23.393559 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:15:23.393527 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeca54a2_94ff_4e9c_9af9_ca286ba63cfd.slice/crio-90ceb308b6424fb3a9606f03a969fc61afebe9226bb55212e94f87b22bee5406 WatchSource:0}: Error finding container 90ceb308b6424fb3a9606f03a969fc61afebe9226bb55212e94f87b22bee5406: Status 404 returned error can't find the container with id 90ceb308b6424fb3a9606f03a969fc61afebe9226bb55212e94f87b22bee5406 Jul 12 15:15:23.396021 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.395954 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:23.396130 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.396058 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:23.396130 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:23.396093 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:24.066078 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:24.066045 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:24.277387 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:24.277354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" event={"ID":"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd","Type":"ContainerStarted","Data":"5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2"} Jul 12 15:15:24.277550 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:24.277419 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" event={"ID":"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd","Type":"ContainerStarted","Data":"90ceb308b6424fb3a9606f03a969fc61afebe9226bb55212e94f87b22bee5406"} Jul 12 15:15:24.299732 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:24.299688 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" podStartSLOduration=2.299673344 podStartE2EDuration="2.299673344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:15:24.297626937 +0000 UTC m=+2072.265510189" watchObservedRunningTime="2026-07-12 15:15:24.299673344 +0000 UTC m=+2072.267556596" Jul 12 15:15:25.257921 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:25.257892 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:25.262464 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:25.262433 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:25.287845 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:25.287808 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:25.288559 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:25.288531 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:26.182387 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:26.182350 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:26.279332 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:26.279301 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:26.423612 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:26.423581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:26.594162 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:26.594085 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:26.965207 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:26.965169 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:27.067518 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:27.067487 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:30.668174 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:30.668140 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:30.984128 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:30.984046 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:31.259780 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.259706 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg"] Jul 12 15:15:31.263122 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.263099 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.265261 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.265241 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e9e47d-openshift-default-dockercfg-8fdl7\"" Jul 12 15:15:31.275361 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.275337 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg"] Jul 12 15:15:31.288079 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.288045 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:31.366396 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366368 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9dm7\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-kube-api-access-z9dm7\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366552 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366419 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-istio-token\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366552 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366502 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-credential-socket\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366645 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-certs\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366645 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366624 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-envoy\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366720 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366661 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-data\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366720 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-socket\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366791 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366747 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e987bb49-55d7-4036-9a1d-828e1912656c-istiod-ca-cert\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.366826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.366801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e987bb49-55d7-4036-9a1d-828e1912656c-istio-podinfo\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468192 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-certs\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468192 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468195 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-envoy\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468406 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-data\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468406 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468249 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-socket\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468406 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468276 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e987bb49-55d7-4036-9a1d-828e1912656c-istiod-ca-cert\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468406 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468311 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e987bb49-55d7-4036-9a1d-828e1912656c-istio-podinfo\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468406 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9dm7\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-kube-api-access-z9dm7\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468406 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468394 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-istio-token\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468700 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468436 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-credential-socket\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468700 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468602 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-certs\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468700 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468676 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-data\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468849 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468725 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-socket\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.468849 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.468751 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-credential-socket\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.469311 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.469292 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e987bb49-55d7-4036-9a1d-828e1912656c-istiod-ca-cert\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.470470 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.470440 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-envoy\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.470939 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.470918 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e987bb49-55d7-4036-9a1d-828e1912656c-istio-podinfo\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.475480 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.475460 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-istio-token\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.475559 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.475521 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9dm7\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-kube-api-access-z9dm7\") pod \"e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.574849 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.574761 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:31.608304 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.608255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:31.710042 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.706529 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg"] Jul 12 15:15:31.710042 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:15:31.707497 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode987bb49_55d7_4036_9a1d_828e1912656c.slice/crio-dcbd88aeee5c5b9cf1a8385bb2a87670e96195e0f1d9910f075a4a1f12c56c3d WatchSource:0}: Error finding container dcbd88aeee5c5b9cf1a8385bb2a87670e96195e0f1d9910f075a4a1f12c56c3d: Status 404 returned error can't find the container with id dcbd88aeee5c5b9cf1a8385bb2a87670e96195e0f1d9910f075a4a1f12c56c3d Jul 12 15:15:31.710042 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.709530 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:31.710042 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.709658 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:31.710042 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:31.709743 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:32.324303 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:32.324264 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" event={"ID":"e987bb49-55d7-4036-9a1d-828e1912656c","Type":"ContainerStarted","Data":"14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea"} Jul 12 15:15:32.324481 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:32.324328 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" event={"ID":"e987bb49-55d7-4036-9a1d-828e1912656c","Type":"ContainerStarted","Data":"dcbd88aeee5c5b9cf1a8385bb2a87670e96195e0f1d9910f075a4a1f12c56c3d"} Jul 12 15:15:32.343597 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:32.343546 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" podStartSLOduration=1.343531491 podStartE2EDuration="1.343531491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:15:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:15:32.341143129 +0000 UTC m=+2080.309026381" watchObservedRunningTime="2026-07-12 15:15:32.343531491 +0000 UTC m=+2080.311414743" Jul 12 15:15:32.575870 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:32.575802 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:32.579976 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:32.579954 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:33.329434 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:33.329398 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:33.330354 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:33.330332 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:34.229161 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:34.229116 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:34.328762 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:34.328725 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:34.426686 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:34.426651 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:34.648523 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:34.648490 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:34.899999 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:34.899918 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:35.000386 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:35.000354 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:38.704368 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:38.704330 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:40.798536 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:40.797457 2573 patch_prober.go:28] interesting pod/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:41450->10.132.0.52:15021: read: connection reset by peer" start-of-body= Jul 12 15:15:40.798536 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:40.797536 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:41450->10.132.0.52:15021: read: connection reset by peer" Jul 12 15:15:41.378832 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:41.378798 2573 generic.go:356] "Generic (PLEG): container finished" podID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerID="5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2" exitCode=0 Jul 12 15:15:41.379050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:41.378890 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" event={"ID":"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd","Type":"ContainerDied","Data":"5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2"} Jul 12 15:15:41.379330 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:41.379314 2573 scope.go:117] "RemoveContainer" containerID="5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2" Jul 12 15:15:41.380310 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:41.380282 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:41.380368 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:41.380353 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:41.380403 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:41.380384 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:42.386686 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:42.386649 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" event={"ID":"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd","Type":"ContainerStarted","Data":"a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345"} Jul 12 15:15:43.276633 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:43.276590 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:43.276852 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:43.276782 2573 patch_prober.go:28] interesting pod/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 15:15:43.276852 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:43.276831 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 15:15:44.199327 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:44.199295 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:44.258437 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:44.258000 2573 patch_prober.go:28] interesting pod/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 15:15:44.258437 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:44.258078 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 15:15:44.299486 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:44.299455 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:45.257826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:45.257775 2573 patch_prober.go:28] interesting pod/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 15:15:45.258408 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:45.257872 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 15:15:46.258778 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:46.258736 2573 patch_prober.go:28] interesting pod/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 15:15:46.259915 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:46.259497 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 15:15:47.261826 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.261736 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:47.262242 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.261996 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:47.262649 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.262634 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:15:47.417346 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.417314 2573 generic.go:356] "Generic (PLEG): container finished" podID="e987bb49-55d7-4036-9a1d-828e1912656c" containerID="14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea" exitCode=0 Jul 12 15:15:47.417524 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.417388 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" event={"ID":"e987bb49-55d7-4036-9a1d-828e1912656c","Type":"ContainerDied","Data":"14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea"} Jul 12 15:15:47.417847 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.417830 2573 scope.go:117] "RemoveContainer" containerID="14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea" Jul 12 15:15:47.418818 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.418786 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:47.418932 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.418875 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:47.418932 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.418921 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:15:47.671591 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.671560 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:47.774928 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:47.774895 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:48.329793 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:48.329747 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:48.430109 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:48.430071 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" event={"ID":"e987bb49-55d7-4036-9a1d-828e1912656c","Type":"ContainerStarted","Data":"3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb"} Jul 12 15:15:48.576852 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:48.576815 2573 patch_prober.go:28] interesting pod/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 15:15:48.577145 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:48.577116 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 15:15:48.581043 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:48.580954 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:49.385277 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.385245 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb"] Jul 12 15:15:49.387540 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.387523 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.390471 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.390445 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-43c370\"/\"kube-root-ca.crt\"" Jul 12 15:15:49.390586 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.390562 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-43c370\"/\"openshift-service-ca.crt\"" Jul 12 15:15:49.390695 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.390676 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-43c370\"/\"test-model-43c370-kserve-self-signed-certs\"" Jul 12 15:15:49.390762 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.390697 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-43c370\"/\"default-dockercfg-67d8g\"" Jul 12 15:15:49.397405 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.397379 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb"] Jul 12 15:15:49.413121 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.413095 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:49.557257 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.557211 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkk2\" (UniqueName: \"kubernetes.io/projected/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-kube-api-access-vpkk2\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.557438 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.557353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-model-cache\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.557528 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.557439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-home\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.557528 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.557496 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tls-certs\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.557641 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.557540 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tmp-dir\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.557641 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.557587 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-dshm\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.575500 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.575455 2573 patch_prober.go:28] interesting pod/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 15:15:49.575693 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.575538 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 15:15:49.659318 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.659209 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-model-cache\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.659621 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.659574 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-home\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.659809 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.659791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tls-certs\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.659948 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.659931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tmp-dir\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.660104 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.660086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-dshm\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.660237 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.660211 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tmp-dir\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.660401 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.660245 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkk2\" (UniqueName: \"kubernetes.io/projected/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-kube-api-access-vpkk2\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.660559 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.659712 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-model-cache\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.660642 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.659948 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-home\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.662988 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.662964 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-dshm\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.665159 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.665131 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tls-certs\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.671649 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.671626 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkk2\" (UniqueName: \"kubernetes.io/projected/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-kube-api-access-vpkk2\") pod \"test-model-43c370-kserve-77787767f7-rqkjb\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.697994 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.697964 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:49.845426 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:49.845365 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb"] Jul 12 15:15:49.846466 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:15:49.846442 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3c19e2d_93fd_46cc_a7d2_3f73030a5a85.slice/crio-8afed0fc450b06b0e3d47c2e6bceb122b12ec63b26792c26013c76c51e8376c9 WatchSource:0}: Error finding container 8afed0fc450b06b0e3d47c2e6bceb122b12ec63b26792c26013c76c51e8376c9: Status 404 returned error can't find the container with id 8afed0fc450b06b0e3d47c2e6bceb122b12ec63b26792c26013c76c51e8376c9 Jul 12 15:15:50.441829 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:50.441785 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" event={"ID":"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85","Type":"ContainerStarted","Data":"3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4"} Jul 12 15:15:50.442245 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:50.441839 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" event={"ID":"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85","Type":"ContainerStarted","Data":"8afed0fc450b06b0e3d47c2e6bceb122b12ec63b26792c26013c76c51e8376c9"} Jul 12 15:15:50.460846 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:50.460783 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" podStartSLOduration=1.460750559 podStartE2EDuration="1.460750559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:15:50.459036332 +0000 UTC m=+2098.426919585" watchObservedRunningTime="2026-07-12 15:15:50.460750559 +0000 UTC m=+2098.428633814" Jul 12 15:15:50.576583 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:50.576126 2573 patch_prober.go:28] interesting pod/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 15:15:50.576583 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:50.576188 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 15:15:51.575641 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:51.575601 2573 patch_prober.go:28] interesting pod/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 15:15:51.576276 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:51.576245 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 15:15:52.579552 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:52.579523 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:52.579910 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:52.579817 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:52.580402 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:52.580384 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:15:52.663565 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:52.663533 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:15:52.669076 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:52.669053 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:15:52.670553 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:52.670532 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:15:52.675617 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:52.675602 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:15:55.617509 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:55.617468 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:55.721500 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:55.721463 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:15:59.698760 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:59.698717 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:59.699337 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:59.698774 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:15:59.711854 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:15:59.711824 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:16:00.506081 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:00.506043 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:16:18.914039 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:18.913980 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:37.110856 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:37.110815 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb"] Jul 12 15:16:37.131934 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:37.131904 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:38.001184 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:38.001143 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:38.166314 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:38.166282 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:38.559750 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:38.559715 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:38.659972 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:38.659941 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:38.761155 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:38.761117 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:38.860260 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:38.860158 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:43.638074 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:43.638034 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:16:52.130927 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.130839 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" podUID="e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" containerName="main" containerID="cri-o://3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4" gracePeriod=45 Jul 12 15:16:52.378472 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.378442 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:16:52.411343 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411275 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-dshm\") pod \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " Jul 12 15:16:52.411343 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411310 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tls-certs\") pod \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " Jul 12 15:16:52.411343 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411333 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tmp-dir\") pod \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " Jul 12 15:16:52.411585 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411354 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-model-cache\") pod \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " Jul 12 15:16:52.411585 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411407 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-home\") pod \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " Jul 12 15:16:52.411585 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411498 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpkk2\" (UniqueName: \"kubernetes.io/projected/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-kube-api-access-vpkk2\") pod \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\" (UID: \"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85\") " Jul 12 15:16:52.411759 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411586 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" (UID: "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:16:52.411759 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411635 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" (UID: "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:16:52.411759 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411739 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-home" (OuterVolumeSpecName: "home") pod "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" (UID: "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:16:52.411910 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411865 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tmp-dir\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:16:52.411910 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411886 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-model-cache\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:16:52.411910 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.411900 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-home\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:16:52.413485 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.413456 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-dshm" (OuterVolumeSpecName: "dshm") pod "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" (UID: "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:16:52.413601 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.413550 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" (UID: "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:16:52.413601 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.413566 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-kube-api-access-vpkk2" (OuterVolumeSpecName: "kube-api-access-vpkk2") pod "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" (UID: "e3c19e2d-93fd-46cc-a7d2-3f73030a5a85"). InnerVolumeSpecName "kube-api-access-vpkk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:16:52.512866 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.512837 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpkk2\" (UniqueName: \"kubernetes.io/projected/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-kube-api-access-vpkk2\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:16:52.512866 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.512862 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-dshm\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:16:52.512866 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.512871 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85-tls-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:16:52.766740 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.766662 2573 generic.go:356] "Generic (PLEG): container finished" podID="e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" containerID="3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4" exitCode=0 Jul 12 15:16:52.766740 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.766709 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" Jul 12 15:16:52.766929 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.766743 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" event={"ID":"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85","Type":"ContainerDied","Data":"3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4"} Jul 12 15:16:52.766929 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.766790 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb" event={"ID":"e3c19e2d-93fd-46cc-a7d2-3f73030a5a85","Type":"ContainerDied","Data":"8afed0fc450b06b0e3d47c2e6bceb122b12ec63b26792c26013c76c51e8376c9"} Jul 12 15:16:52.766929 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.766807 2573 scope.go:117] "RemoveContainer" containerID="3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4" Jul 12 15:16:52.775517 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.775495 2573 scope.go:117] "RemoveContainer" containerID="3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4" Jul 12 15:16:52.775756 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:16:52.775736 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4\": container with ID starting with 3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4 not found: ID does not exist" containerID="3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4" Jul 12 15:16:52.775796 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.775766 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4"} err="failed to get container status \"3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4\": rpc error: code = NotFound desc = could not find container \"3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4\": container with ID starting with 3bd89dd9e1e494d73cdebc749a43899f1651f80fe965f6a7ad55527b799592e4 not found: ID does not exist" Jul 12 15:16:52.782138 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.782114 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb"] Jul 12 15:16:52.785596 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:52.785576 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-43c370/test-model-43c370-kserve-77787767f7-rqkjb"] Jul 12 15:16:54.576382 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:16:54.576347 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" path="/var/lib/kubelet/pods/e3c19e2d-93fd-46cc-a7d2-3f73030a5a85/volumes" Jul 12 15:17:20.274189 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:20.274157 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:20.527550 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:20.527470 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm"] Jul 12 15:17:20.527757 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:20.527721 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" containerID="cri-o://a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345" gracePeriod=30 Jul 12 15:17:20.551702 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:20.551651 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:20.932991 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:20.932954 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:21.346209 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:21.346128 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:21.479763 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:21.479733 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:21.582878 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:21.582845 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:21.752200 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:21.752170 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:25.778484 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.778462 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:17:25.901068 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901037 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-748wr\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-kube-api-access-748wr\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901217 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901099 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-certs\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901217 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901130 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-data\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901217 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901160 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-socket\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901217 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901177 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-podinfo\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901217 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901209 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-token\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901260 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-credential-socket\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901288 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istiod-ca-cert\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901311 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-envoy\") pod \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\" (UID: \"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd\") " Jul 12 15:17:25.901469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901413 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:25.901679 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901481 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:25.901737 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901683 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:25.901737 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901702 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:25.901839 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901816 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:25.901895 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.901859 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:17:25.902069 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.902049 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-data" (OuterVolumeSpecName: "istio-data") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:25.903407 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.903385 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:17:25.903506 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.903428 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-token" (OuterVolumeSpecName: "istio-token") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:17:25.903641 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.903621 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:25.903690 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.903629 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-kube-api-access-748wr" (OuterVolumeSpecName: "kube-api-access-748wr") pod "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" (UID: "eeca54a2-94ff-4e9c-9af9-ca286ba63cfd"). InnerVolumeSpecName "kube-api-access-748wr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:17:25.947595 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.947573 2573 generic.go:356] "Generic (PLEG): container finished" podID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerID="a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345" exitCode=0 Jul 12 15:17:25.947698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.947627 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" Jul 12 15:17:25.947762 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.947623 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" event={"ID":"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd","Type":"ContainerDied","Data":"a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345"} Jul 12 15:17:25.947762 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.947747 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm" event={"ID":"eeca54a2-94ff-4e9c-9af9-ca286ba63cfd","Type":"ContainerDied","Data":"90ceb308b6424fb3a9606f03a969fc61afebe9226bb55212e94f87b22bee5406"} Jul 12 15:17:25.947855 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.947765 2573 scope.go:117] "RemoveContainer" containerID="a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345" Jul 12 15:17:25.959149 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.959130 2573 scope.go:117] "RemoveContainer" containerID="5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2" Jul 12 15:17:25.967201 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.967184 2573 scope.go:117] "RemoveContainer" containerID="a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345" Jul 12 15:17:25.967432 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:17:25.967414 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345\": container with ID starting with a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345 not found: ID does not exist" containerID="a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345" Jul 12 15:17:25.967503 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.967445 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345"} err="failed to get container status \"a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345\": rpc error: code = NotFound desc = could not find container \"a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345\": container with ID starting with a9ffdac8945271564ff553890ca0c29f259e56319317dac743598a00903c2345 not found: ID does not exist" Jul 12 15:17:25.967503 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.967467 2573 scope.go:117] "RemoveContainer" containerID="5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2" Jul 12 15:17:25.967704 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:17:25.967687 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2\": container with ID starting with 5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2 not found: ID does not exist" containerID="5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2" Jul 12 15:17:25.967744 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.967709 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2"} err="failed to get container status \"5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2\": rpc error: code = NotFound desc = could not find container \"5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2\": container with ID starting with 5d54d4e77028f7134cfa231f66e4162552ae51ea4cf3c036a901bfa906caa7b2 not found: ID does not exist" Jul 12 15:17:25.971652 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.971631 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm"] Jul 12 15:17:25.975780 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:25.975760 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-43c370-openshift-default-776f47d4bf-bwdvm"] Jul 12 15:17:26.002302 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.002282 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:26.002302 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.002301 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:26.002416 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.002310 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:26.002416 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.002320 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:26.002416 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.002328 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-748wr\" (UniqueName: \"kubernetes.io/projected/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-kube-api-access-748wr\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:26.002416 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.002336 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:26.002416 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.002344 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:26.576274 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:26.576246 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" path="/var/lib/kubelet/pods/eeca54a2-94ff-4e9c-9af9-ca286ba63cfd/volumes" Jul 12 15:17:27.141150 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:27.141107 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:27.200651 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:27.200621 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:27.381893 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:27.381862 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg"] Jul 12 15:17:27.382145 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:27.382115 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" containerID="cri-o://3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb" gracePeriod=30 Jul 12 15:17:27.403506 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:27.403405 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:27.961929 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:27.961884 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:28.261372 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:28.261290 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:28.628934 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:28.628868 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:30.959798 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:30.959752 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:31.045379 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:31.045344 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:31.107004 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:31.106975 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:31.296490 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:31.296409 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:31.598046 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:31.597942 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:31.703105 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:31.703078 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:32.628295 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.628269 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:17:32.765677 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765589 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-istio-token\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765677 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765626 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-envoy\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765677 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765656 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-credential-socket\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765773 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-data\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765828 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e987bb49-55d7-4036-9a1d-828e1912656c-istiod-ca-cert\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765877 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9dm7\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-kube-api-access-z9dm7\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765909 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e987bb49-55d7-4036-9a1d-828e1912656c-istio-podinfo\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765914 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:32.765966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765941 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-certs\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.765966 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.765967 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-socket\") pod \"e987bb49-55d7-4036-9a1d-828e1912656c\" (UID: \"e987bb49-55d7-4036-9a1d-828e1912656c\") " Jul 12 15:17:32.766359 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.766236 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e987bb49-55d7-4036-9a1d-828e1912656c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:17:32.766469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.766446 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:32.766567 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.766530 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:32.766704 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.766613 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-data" (OuterVolumeSpecName: "istio-data") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:32.767870 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.767847 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:17:32.767975 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.767956 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e987bb49-55d7-4036-9a1d-828e1912656c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:17:32.768205 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.768190 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:17:32.768273 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.768258 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-kube-api-access-z9dm7" (OuterVolumeSpecName: "kube-api-access-z9dm7") pod "e987bb49-55d7-4036-9a1d-828e1912656c" (UID: "e987bb49-55d7-4036-9a1d-828e1912656c"). InnerVolumeSpecName "kube-api-access-z9dm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:17:32.866933 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866910 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9dm7\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-kube-api-access-z9dm7\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.866933 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866931 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e987bb49-55d7-4036-9a1d-828e1912656c-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.867097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866941 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.867097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866949 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.867097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866957 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e987bb49-55d7-4036-9a1d-828e1912656c-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.867097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866965 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.867097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866973 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.867097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866980 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e987bb49-55d7-4036-9a1d-828e1912656c-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.867097 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.866988 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e987bb49-55d7-4036-9a1d-828e1912656c-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:17:32.986540 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.986506 2573 generic.go:356] "Generic (PLEG): container finished" podID="e987bb49-55d7-4036-9a1d-828e1912656c" containerID="3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb" exitCode=0 Jul 12 15:17:32.986758 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.986566 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" Jul 12 15:17:32.986758 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.986599 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" event={"ID":"e987bb49-55d7-4036-9a1d-828e1912656c","Type":"ContainerDied","Data":"3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb"} Jul 12 15:17:32.986758 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.986668 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg" event={"ID":"e987bb49-55d7-4036-9a1d-828e1912656c","Type":"ContainerDied","Data":"dcbd88aeee5c5b9cf1a8385bb2a87670e96195e0f1d9910f075a4a1f12c56c3d"} Jul 12 15:17:32.986758 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.986703 2573 scope.go:117] "RemoveContainer" containerID="3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb" Jul 12 15:17:32.996306 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:32.996277 2573 scope.go:117] "RemoveContainer" containerID="14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea" Jul 12 15:17:33.003758 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:33.003744 2573 scope.go:117] "RemoveContainer" containerID="3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb" Jul 12 15:17:33.003993 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:17:33.003972 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb\": container with ID starting with 3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb not found: ID does not exist" containerID="3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb" Jul 12 15:17:33.004080 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:33.004004 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb"} err="failed to get container status \"3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb\": rpc error: code = NotFound desc = could not find container \"3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb\": container with ID starting with 3b658173d696ebd3d229e4086f6a71a554ef57a535ffc2702e6c1d7d9f72dbbb not found: ID does not exist" Jul 12 15:17:33.004080 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:33.004056 2573 scope.go:117] "RemoveContainer" containerID="14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea" Jul 12 15:17:33.004311 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:17:33.004293 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea\": container with ID starting with 14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea not found: ID does not exist" containerID="14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea" Jul 12 15:17:33.004347 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:33.004318 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea"} err="failed to get container status \"14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea\": rpc error: code = NotFound desc = could not find container \"14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea\": container with ID starting with 14c0e25050873f1414058621ed191be7c9e3025fa5e9df50e69f75d6689b2eea not found: ID does not exist" Jul 12 15:17:33.008904 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:33.008884 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg"] Jul 12 15:17:33.012560 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:33.012541 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e9e47d-openshift-default-6d8d7d7987-b44dg"] Jul 12 15:17:34.576960 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:34.576927 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" path="/var/lib/kubelet/pods/e987bb49-55d7-4036-9a1d-828e1912656c/volumes" Jul 12 15:17:35.423358 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:35.423320 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:35.864312 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:35.864223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:36.185206 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185173 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s"] Jul 12 15:17:36.185874 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185847 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" Jul 12 15:17:36.185874 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185872 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185890 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" containerName="main" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185898 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" containerName="main" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185910 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185918 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185948 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185956 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185968 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" Jul 12 15:17:36.186050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.185975 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" Jul 12 15:17:36.186337 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.186065 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" Jul 12 15:17:36.186337 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.186078 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" Jul 12 15:17:36.186337 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.186089 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeca54a2-94ff-4e9c-9af9-ca286ba63cfd" containerName="istio-proxy" Jul 12 15:17:36.186337 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.186097 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c19e2d-93fd-46cc-a7d2-3f73030a5a85" containerName="main" Jul 12 15:17:36.190992 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.190971 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.193151 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.193121 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-05f888-openshift-default-dockercfg-m8b5b\"" Jul 12 15:17:36.201756 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.201734 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s"] Jul 12 15:17:36.213643 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.213623 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:36.298000 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.297964 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-certs\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298219 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298043 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-socket\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298219 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298068 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-data\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298219 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-podinfo\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298219 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298104 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-token\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298219 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298153 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crrxf\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-kube-api-access-crrxf\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298219 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298169 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-envoy\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298529 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298223 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-credential-socket\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.298529 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.298268 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istiod-ca-cert\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.373525 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.373495 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:36.398832 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.398800 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-certs\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.398978 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.398853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-socket\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.398978 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.398876 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-data\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.398978 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.398895 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-podinfo\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.398978 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.398911 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-token\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399202 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crrxf\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-kube-api-access-crrxf\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399202 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-envoy\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399202 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399138 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-credential-socket\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399202 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399181 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istiod-ca-cert\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399396 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399236 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-certs\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399396 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399277 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-data\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399503 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399413 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-socket\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399552 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399506 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-credential-socket\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.399902 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.399878 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istiod-ca-cert\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.401215 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.401194 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-envoy\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.401360 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.401344 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-podinfo\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.406551 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.406529 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-token\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.406633 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.406623 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crrxf\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-kube-api-access-crrxf\") pod \"e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.502203 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.502114 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:36.634766 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.634734 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s"] Jul 12 15:17:36.636267 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:17:36.636236 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33631a2e_0b0d_4dd8_a013_fc8c1e892775.slice/crio-045976e726b4d239d759a7e3c4cdf74f096d411bd462527475ca61d6bf732207 WatchSource:0}: Error finding container 045976e726b4d239d759a7e3c4cdf74f096d411bd462527475ca61d6bf732207: Status 404 returned error can't find the container with id 045976e726b4d239d759a7e3c4cdf74f096d411bd462527475ca61d6bf732207 Jul 12 15:17:36.638626 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.638593 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:17:36.638715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.638654 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:17:36.638715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:36.638683 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:17:37.009755 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:37.009723 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" event={"ID":"33631a2e-0b0d-4dd8-a013-fc8c1e892775","Type":"ContainerStarted","Data":"830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163"} Jul 12 15:17:37.010261 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:37.009772 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" event={"ID":"33631a2e-0b0d-4dd8-a013-fc8c1e892775","Type":"ContainerStarted","Data":"045976e726b4d239d759a7e3c4cdf74f096d411bd462527475ca61d6bf732207"} Jul 12 15:17:37.031268 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:37.031217 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" podStartSLOduration=1.031198432 podStartE2EDuration="1.031198432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:17:37.029852295 +0000 UTC m=+2204.997735559" watchObservedRunningTime="2026-07-12 15:17:37.031198432 +0000 UTC m=+2204.999081700" Jul 12 15:17:37.502798 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:37.502765 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:37.504287 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:37.504258 2573 patch_prober.go:28] interesting pod/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 15:17:37.504415 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:37.504315 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 15:17:38.506701 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:38.506671 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:38.928035 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:38.927994 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:39.014338 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:39.014301 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:39.020821 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:39.020791 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:39.021865 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:39.021843 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:39.081885 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:39.081854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:39.271339 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:39.271255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:39.579174 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:39.579091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:39.680328 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:39.680297 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:43.379278 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:43.379238 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:50.082451 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:50.082416 2573 generic.go:356] "Generic (PLEG): container finished" podID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerID="830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163" exitCode=0 Jul 12 15:17:50.082890 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:50.082489 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" event={"ID":"33631a2e-0b0d-4dd8-a013-fc8c1e892775","Type":"ContainerDied","Data":"830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163"} Jul 12 15:17:50.082890 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:50.082870 2573 scope.go:117] "RemoveContainer" containerID="830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163" Jul 12 15:17:50.083846 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:50.083820 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:17:50.083895 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:50.083886 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:17:50.083929 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:50.083920 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:17:51.090318 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.090283 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" event={"ID":"33631a2e-0b0d-4dd8-a013-fc8c1e892775","Type":"ContainerStarted","Data":"b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3"} Jul 12 15:17:51.385507 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.385477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:51.502875 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.502836 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:51.504205 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.504177 2573 patch_prober.go:28] interesting pod/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 15:17:51.504308 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.504236 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 15:17:51.761088 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.760943 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:51.860289 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.860255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:51.960151 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:51.960102 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:52.081673 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:52.081570 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:52.503192 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:52.503161 2573 patch_prober.go:28] interesting pod/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 15:17:52.503579 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:52.503210 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 15:17:53.503420 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:53.503382 2573 patch_prober.go:28] interesting pod/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 15:17:53.503804 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:53.503438 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 15:17:54.021575 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:54.021548 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:54.503037 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:54.502975 2573 patch_prober.go:28] interesting pod/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 15:17:54.503220 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:54.503053 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 15:17:55.506129 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:55.506096 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:55.507200 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:55.507179 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:17:57.365135 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:57.365090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:57.419830 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:57.419800 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:57.612088 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:57.612052 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:58.087636 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:58.087603 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:58.379900 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:58.379869 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:58.591689 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:58.591656 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:58.693598 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:58.693518 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:17:58.860536 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:17:58.860508 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:04.100383 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:04.100347 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:04.165796 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:04.165768 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:04.333548 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:04.333517 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s"] Jul 12 15:18:04.333791 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:04.333756 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" containerID="cri-o://b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3" gracePeriod=30 Jul 12 15:18:04.354424 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:04.354365 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:04.892909 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:04.892877 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:05.193795 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:05.193715 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:05.376922 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:05.376881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:08.171337 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:08.171299 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:08.256667 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:08.256626 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:08.318797 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:08.318762 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:08.419253 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:08.419218 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:08.732255 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:08.732216 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:08.832663 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:08.832626 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:09.584079 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.584052 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:18:09.691428 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691387 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-data\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691428 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691429 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-socket\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691466 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-envoy\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691512 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-credential-socket\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691556 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-token\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691579 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-certs\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691633 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-podinfo\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691663 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crrxf\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-kube-api-access-crrxf\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691678 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:18:09.691698 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691698 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istiod-ca-cert\") pod \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\" (UID: \"33631a2e-0b0d-4dd8-a013-fc8c1e892775\") " Jul 12 15:18:09.692080 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691925 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:18:09.692080 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.691978 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.692297 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.692274 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-data" (OuterVolumeSpecName: "istio-data") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:18:09.692385 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.692229 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:18:09.692385 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.692329 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:18:09.693902 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.693876 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:18:09.694025 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.693901 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-kube-api-access-crrxf" (OuterVolumeSpecName: "kube-api-access-crrxf") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "kube-api-access-crrxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:18:09.694104 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.694080 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:18:09.694178 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.694163 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-token" (OuterVolumeSpecName: "istio-token") pod "33631a2e-0b0d-4dd8-a013-fc8c1e892775" (UID: "33631a2e-0b0d-4dd8-a013-fc8c1e892775"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:18:09.792839 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792802 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.792839 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792832 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.792839 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792842 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.792839 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792850 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.793157 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792858 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.793157 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792865 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crrxf\" (UniqueName: \"kubernetes.io/projected/33631a2e-0b0d-4dd8-a013-fc8c1e892775-kube-api-access-crrxf\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.793157 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792873 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:09.793157 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:09.792880 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33631a2e-0b0d-4dd8-a013-fc8c1e892775-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:18:10.086861 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.086774 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:10.139555 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.139525 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:10.192822 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.192783 2573 generic.go:356] "Generic (PLEG): container finished" podID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerID="b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3" exitCode=0 Jul 12 15:18:10.192959 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.192839 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" Jul 12 15:18:10.192959 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.192870 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" event={"ID":"33631a2e-0b0d-4dd8-a013-fc8c1e892775","Type":"ContainerDied","Data":"b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3"} Jul 12 15:18:10.192959 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.192922 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s" event={"ID":"33631a2e-0b0d-4dd8-a013-fc8c1e892775","Type":"ContainerDied","Data":"045976e726b4d239d759a7e3c4cdf74f096d411bd462527475ca61d6bf732207"} Jul 12 15:18:10.192959 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.192943 2573 scope.go:117] "RemoveContainer" containerID="b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3" Jul 12 15:18:10.202924 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.202905 2573 scope.go:117] "RemoveContainer" containerID="830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163" Jul 12 15:18:10.211275 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.211255 2573 scope.go:117] "RemoveContainer" containerID="b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3" Jul 12 15:18:10.211544 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:18:10.211523 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3\": container with ID starting with b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3 not found: ID does not exist" containerID="b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3" Jul 12 15:18:10.211617 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.211558 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3"} err="failed to get container status \"b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3\": rpc error: code = NotFound desc = could not find container \"b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3\": container with ID starting with b9a3cf467069aabcdbfbd8e42954842466a1baa14c2951bc339267117b736bc3 not found: ID does not exist" Jul 12 15:18:10.211617 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.211586 2573 scope.go:117] "RemoveContainer" containerID="830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163" Jul 12 15:18:10.211851 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:18:10.211833 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163\": container with ID starting with 830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163 not found: ID does not exist" containerID="830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163" Jul 12 15:18:10.211891 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.211858 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163"} err="failed to get container status \"830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163\": rpc error: code = NotFound desc = could not find container \"830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163\": container with ID starting with 830c372a6f68cad1d7ccfc297242d897639c2476d53ab115b8783e69ccae3163 not found: ID does not exist" Jul 12 15:18:10.221731 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.221709 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s"] Jul 12 15:18:10.225351 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.225329 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-05f888-openshift-default-b9b58dd99-4kk7s"] Jul 12 15:18:10.575940 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:10.575901 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" path="/var/lib/kubelet/pods/33631a2e-0b0d-4dd8-a013-fc8c1e892775/volumes" Jul 12 15:18:12.647083 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:12.647042 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:13.008874 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.008789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:13.227840 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.227802 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw"] Jul 12 15:18:13.228835 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.228800 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" Jul 12 15:18:13.228835 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.228834 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" Jul 12 15:18:13.229048 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.228873 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" Jul 12 15:18:13.229048 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.228882 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" Jul 12 15:18:13.229156 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.229073 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" Jul 12 15:18:13.229156 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.229095 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e987bb49-55d7-4036-9a1d-828e1912656c" containerName="istio-proxy" Jul 12 15:18:13.235028 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.234985 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.237484 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.237461 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-69f681-openshift-default-dockercfg-bgtqv\"" Jul 12 15:18:13.238578 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.238551 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw"] Jul 12 15:18:13.254090 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.254065 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:13.324981 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.324902 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-envoy\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.324981 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.324943 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-certs\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.324981 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.324962 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-data\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.325223 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.324984 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7w2d\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-kube-api-access-c7w2d\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.325223 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.325090 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-socket\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.325223 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.325129 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-credential-socket\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.325223 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.325157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-istio-token\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.325223 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.325189 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1600549-f2bd-4605-8afb-dc75609cba19-istio-podinfo\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.325223 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.325214 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1600549-f2bd-4605-8afb-dc75609cba19-istiod-ca-cert\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426330 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426295 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-envoy\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426494 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426338 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-certs\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426494 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426360 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-data\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426494 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426391 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7w2d\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-kube-api-access-c7w2d\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426494 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-socket\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426690 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-credential-socket\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426690 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426618 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-istio-token\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426690 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426649 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1600549-f2bd-4605-8afb-dc75609cba19-istio-podinfo\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426690 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426675 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1600549-f2bd-4605-8afb-dc75609cba19-istiod-ca-cert\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426888 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426776 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-certs\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426888 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426839 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-socket\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.426998 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.426975 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-data\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.427076 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.427024 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-credential-socket\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.427373 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.427351 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1600549-f2bd-4605-8afb-dc75609cba19-istiod-ca-cert\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.428585 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.428561 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-envoy\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.429050 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.429034 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1600549-f2bd-4605-8afb-dc75609cba19-istio-podinfo\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.433423 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.433401 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7w2d\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-kube-api-access-c7w2d\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.433518 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.433500 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-istio-token\") pod \"e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.546488 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.546450 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:13.680174 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.680096 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw"] Jul 12 15:18:13.683717 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:18:13.683686 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1600549_f2bd_4605_8afb_dc75609cba19.slice/crio-febfbf7c9136893cbc62fdb30e5dc5b2550c21bc961f8d908251b31bb055d1a9 WatchSource:0}: Error finding container febfbf7c9136893cbc62fdb30e5dc5b2550c21bc961f8d908251b31bb055d1a9: Status 404 returned error can't find the container with id febfbf7c9136893cbc62fdb30e5dc5b2550c21bc961f8d908251b31bb055d1a9 Jul 12 15:18:13.686160 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.686118 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:18:13.686252 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.686202 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:18:13.686252 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.686244 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:18:13.789238 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:13.789201 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:14.216576 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:14.216542 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" event={"ID":"d1600549-f2bd-4605-8afb-dc75609cba19","Type":"ContainerStarted","Data":"9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8"} Jul 12 15:18:14.216763 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:14.216595 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" event={"ID":"d1600549-f2bd-4605-8afb-dc75609cba19","Type":"ContainerStarted","Data":"febfbf7c9136893cbc62fdb30e5dc5b2550c21bc961f8d908251b31bb055d1a9"} Jul 12 15:18:14.237509 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:14.237463 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" podStartSLOduration=1.237447377 podStartE2EDuration="1.237447377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:18:14.235679464 +0000 UTC m=+2242.203562716" watchObservedRunningTime="2026-07-12 15:18:14.237447377 +0000 UTC m=+2242.205330629" Jul 12 15:18:14.546829 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:14.546740 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:14.551769 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:14.551746 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:15.222153 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:15.222122 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:15.223140 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:15.223120 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:16.093312 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:16.093262 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:16.183493 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:16.183448 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:16.249345 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:16.249311 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:16.767672 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:16.767638 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:16.863451 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:16.863419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:17.140727 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:17.140693 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:20.606471 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:20.606434 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:27.292647 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:27.292557 2573 generic.go:356] "Generic (PLEG): container finished" podID="d1600549-f2bd-4605-8afb-dc75609cba19" containerID="9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8" exitCode=0 Jul 12 15:18:27.293205 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:27.292634 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" event={"ID":"d1600549-f2bd-4605-8afb-dc75609cba19","Type":"ContainerDied","Data":"9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8"} Jul 12 15:18:27.293205 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:27.293161 2573 scope.go:117] "RemoveContainer" containerID="9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8" Jul 12 15:18:27.294213 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:27.294187 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:18:27.294285 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:27.294254 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:18:27.294323 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:27.294288 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 12 15:18:28.299636 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:28.299597 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" event={"ID":"d1600549-f2bd-4605-8afb-dc75609cba19","Type":"ContainerStarted","Data":"79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842"} Jul 12 15:18:28.547069 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:28.547033 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:28.548089 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:28.548064 2573 patch_prober.go:28] interesting pod/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 15:18:28.548210 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:28.548109 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 15:18:29.547664 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:29.547631 2573 patch_prober.go:28] interesting pod/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 15:18:29.548057 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:29.547685 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 15:18:29.582725 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:29.582695 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:29.684512 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:29.684480 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:30.222722 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:30.222685 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:30.547571 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:30.547478 2573 patch_prober.go:28] interesting pod/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 15:18:30.547571 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:30.547538 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 15:18:31.546876 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:31.546844 2573 patch_prober.go:28] interesting pod/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 15:18:31.547065 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:31.546896 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 15:18:32.550543 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:32.550511 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:32.551385 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:32.551363 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:18:37.516630 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:37.516597 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:37.617933 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:37.617902 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:38.715774 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:38.715741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:18:38.817495 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:18:38.817469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:19:06.641513 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:19:06.641477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:19:46.460993 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:19:46.460854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:19:57.529671 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:19:57.529634 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:20:52.702718 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:20:52.702690 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:20:52.708982 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:20:52.708956 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:20:52.709665 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:20:52.709644 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:20:52.716117 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:20:52.716099 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hrksg_2b612495-6f09-40f5-9eb9-b8e603d61d37/ovn-acl-logging/0.log" Jul 12 15:21:11.412167 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:11.412129 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:25.294882 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:25.294795 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:25.696218 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:25.696189 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:25.796344 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:25.796307 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:25.896489 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:25.896454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:25.973973 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:25.973887 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:31.354115 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:31.354082 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:31.412788 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:31.412758 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:31.613390 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:31.613315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:31.999975 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:31.999938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:32.416458 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:32.416429 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:32.512969 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:32.512943 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:32.613773 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:32.613731 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:32.710495 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:32.710422 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:38.032514 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:38.032479 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:38.133302 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:38.133267 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:38.274092 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:38.274059 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw"] Jul 12 15:21:38.274302 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:38.274273 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" containerID="cri-o://79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842" gracePeriod=30 Jul 12 15:21:38.288482 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:38.288423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:39.048645 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:39.048610 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84dcbdc54b-scqgq_a1442e75-527f-4a31-bc6d-ea14bf4f6a13/authorino/0.log" Jul 12 15:21:41.082822 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:41.082791 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:41.138352 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:41.138313 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:43.522141 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.522117 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:21:43.641840 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.641805 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-envoy\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642034 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.641856 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-data\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642034 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.641900 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-istio-token\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642034 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.641927 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-credential-socket\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642034 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.641960 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1600549-f2bd-4605-8afb-dc75609cba19-istio-podinfo\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642034 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.641998 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7w2d\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-kube-api-access-c7w2d\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642301 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642050 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-certs\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642301 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642088 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1600549-f2bd-4605-8afb-dc75609cba19-istiod-ca-cert\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642301 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642121 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-socket\") pod \"d1600549-f2bd-4605-8afb-dc75609cba19\" (UID: \"d1600549-f2bd-4605-8afb-dc75609cba19\") " Jul 12 15:21:43.642464 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642383 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:21:43.642464 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642418 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:21:43.642582 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642535 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:21:43.642765 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642734 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1600549-f2bd-4605-8afb-dc75609cba19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:21:43.642901 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.642875 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:21:43.644276 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.644249 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:21:43.644381 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.644327 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-kube-api-access-c7w2d" (OuterVolumeSpecName: "kube-api-access-c7w2d") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "kube-api-access-c7w2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:21:43.644517 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.644492 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1600549-f2bd-4605-8afb-dc75609cba19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:21:43.644617 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.644500 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1600549-f2bd-4605-8afb-dc75609cba19" (UID: "d1600549-f2bd-4605-8afb-dc75609cba19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:21:43.743543 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743500 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-istio-token\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743543 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743533 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-credential-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743543 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743544 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1600549-f2bd-4605-8afb-dc75609cba19-istio-podinfo\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743543 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743554 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7w2d\" (UniqueName: \"kubernetes.io/projected/d1600549-f2bd-4605-8afb-dc75609cba19-kube-api-access-c7w2d\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743821 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743562 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-certs\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743821 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743571 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1600549-f2bd-4605-8afb-dc75609cba19-istiod-ca-cert\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743821 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743579 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-workload-socket\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743821 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743586 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-envoy\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.743821 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.743595 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1600549-f2bd-4605-8afb-dc75609cba19-istio-data\") on node \"ip-10-0-142-78.ec2.internal\" DevicePath \"\"" Jul 12 15:21:43.792038 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:43.791998 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-56fs9_65758d6f-63fb-4b2a-b915-6fa74894d187/server/0.log" Jul 12 15:21:44.024713 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.024632 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9f5575-88csv_9bb28d76-211c-44e3-9651-07b03bbde796/copy-manifests/0.log" Jul 12 15:21:44.036385 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.036361 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9f5575-88csv_9bb28d76-211c-44e3-9651-07b03bbde796/manager/0.log" Jul 12 15:21:44.277168 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.277078 2573 generic.go:356] "Generic (PLEG): container finished" podID="d1600549-f2bd-4605-8afb-dc75609cba19" containerID="79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842" exitCode=0 Jul 12 15:21:44.277168 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.277128 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" Jul 12 15:21:44.277168 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.277137 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" event={"ID":"d1600549-f2bd-4605-8afb-dc75609cba19","Type":"ContainerDied","Data":"79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842"} Jul 12 15:21:44.277420 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.277179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw" event={"ID":"d1600549-f2bd-4605-8afb-dc75609cba19","Type":"ContainerDied","Data":"febfbf7c9136893cbc62fdb30e5dc5b2550c21bc961f8d908251b31bb055d1a9"} Jul 12 15:21:44.277420 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.277200 2573 scope.go:117] "RemoveContainer" containerID="79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842" Jul 12 15:21:44.286576 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.286557 2573 scope.go:117] "RemoveContainer" containerID="9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8" Jul 12 15:21:44.294330 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.294312 2573 scope.go:117] "RemoveContainer" containerID="79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842" Jul 12 15:21:44.294640 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:21:44.294615 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842\": container with ID starting with 79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842 not found: ID does not exist" containerID="79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842" Jul 12 15:21:44.294748 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.294646 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842"} err="failed to get container status \"79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842\": rpc error: code = NotFound desc = could not find container \"79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842\": container with ID starting with 79415865f4249844e568392dae1844a2b539495b0efe68a675a5ef2a6908e842 not found: ID does not exist" Jul 12 15:21:44.294748 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.294671 2573 scope.go:117] "RemoveContainer" containerID="9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8" Jul 12 15:21:44.294947 ip-10-0-142-78 kubenswrapper[2573]: E0712 15:21:44.294927 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8\": container with ID starting with 9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8 not found: ID does not exist" containerID="9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8" Jul 12 15:21:44.295033 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.294956 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8"} err="failed to get container status \"9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8\": rpc error: code = NotFound desc = could not find container \"9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8\": container with ID starting with 9424984c1f7bc1a386357f6286177b202859fcac1d28bce2c46da21aa08653b8 not found: ID does not exist" Jul 12 15:21:44.297323 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.297303 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw"] Jul 12 15:21:44.303457 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.303435 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-69f681-openshift-default-665c4bb9f9-2rffw"] Jul 12 15:21:44.576235 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:44.576145 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" path="/var/lib/kubelet/pods/d1600549-f2bd-4605-8afb-dc75609cba19/volumes" Jul 12 15:21:45.042400 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.042372 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7_8507da11-9656-4b5f-9f36-105cb2fdae9a/util/0.log" Jul 12 15:21:45.047303 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.047281 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7_8507da11-9656-4b5f-9f36-105cb2fdae9a/pull/0.log" Jul 12 15:21:45.052258 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.052237 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7_8507da11-9656-4b5f-9f36-105cb2fdae9a/extract/0.log" Jul 12 15:21:45.159614 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.159587 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb_309ebb07-a049-4716-a791-e88f343d62d9/util/0.log" Jul 12 15:21:45.164546 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.164527 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb_309ebb07-a049-4716-a791-e88f343d62d9/pull/0.log" Jul 12 15:21:45.169260 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.169242 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb_309ebb07-a049-4716-a791-e88f343d62d9/extract/0.log" Jul 12 15:21:45.275357 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.275333 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49_f48a30dc-8e56-4d30-a219-75661726d3c5/util/0.log" Jul 12 15:21:45.280145 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.280104 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49_f48a30dc-8e56-4d30-a219-75661726d3c5/pull/0.log" Jul 12 15:21:45.284469 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.284451 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49_f48a30dc-8e56-4d30-a219-75661726d3c5/extract/0.log" Jul 12 15:21:45.392400 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.392378 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7_cf2e9876-c6ef-4356-a884-14ec05b98a45/util/0.log" Jul 12 15:21:45.397894 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.397865 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7_cf2e9876-c6ef-4356-a884-14ec05b98a45/pull/0.log" Jul 12 15:21:45.403612 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.403596 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7_cf2e9876-c6ef-4356-a884-14ec05b98a45/extract/0.log" Jul 12 15:21:45.519764 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:45.519732 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84dcbdc54b-scqgq_a1442e75-527f-4a31-bc6d-ea14bf4f6a13/authorino/0.log" Jul 12 15:21:46.652687 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:46.652656 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t6vpt_ee867255-1dd5-465b-81ab-e7f2d4b5d5cb/istio-proxy/0.log" Jul 12 15:21:46.875358 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:46.875325 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b855bfc69-bdqdf_5075b719-f93f-4559-beda-a6c11a335af9/kube-auth-proxy/0.log" Jul 12 15:21:47.880941 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:47.880898 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8rm5b_6bb71cc5-45e8-424f-b595-96dc8ae51606/main/0.log" Jul 12 15:21:48.097737 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:48.097703 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqk6rc_96b99278-616c-4b2a-9693-48505b0349d8/main/0.log" Jul 12 15:21:51.514919 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:51.514884 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:51.561337 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:51.561309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:52.499169 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.499135 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:52.499657 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.499635 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqvj9/must-gather-9m2qj"] Jul 12 15:21:52.499992 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.499976 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" Jul 12 15:21:52.500091 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.499995 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" Jul 12 15:21:52.500091 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.500007 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" Jul 12 15:21:52.500091 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.500029 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" Jul 12 15:21:52.500243 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.500143 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" Jul 12 15:21:52.500243 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.500159 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="33631a2e-0b0d-4dd8-a013-fc8c1e892775" containerName="istio-proxy" Jul 12 15:21:52.506546 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.501137 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1600549-f2bd-4605-8afb-dc75609cba19" containerName="istio-proxy" Jul 12 15:21:52.506546 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.506506 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.508969 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.508948 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqvj9\"/\"kube-root-ca.crt\"" Jul 12 15:21:52.509147 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.509045 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pqvj9\"/\"default-dockercfg-r5xdj\"" Jul 12 15:21:52.509555 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.509520 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqvj9\"/\"openshift-service-ca.crt\"" Jul 12 15:21:52.510612 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.510586 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvj9/must-gather-9m2qj"] Jul 12 15:21:52.601076 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.601048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:52.625741 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.625714 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0a54de14-beb0-4721-a638-2d15eb4a52a1-must-gather-output\") pod \"must-gather-9m2qj\" (UID: \"0a54de14-beb0-4721-a638-2d15eb4a52a1\") " pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.625855 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.625830 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7rfx\" (UniqueName: \"kubernetes.io/projected/0a54de14-beb0-4721-a638-2d15eb4a52a1-kube-api-access-l7rfx\") pod \"must-gather-9m2qj\" (UID: \"0a54de14-beb0-4721-a638-2d15eb4a52a1\") " pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.726532 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.726508 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7rfx\" (UniqueName: \"kubernetes.io/projected/0a54de14-beb0-4721-a638-2d15eb4a52a1-kube-api-access-l7rfx\") pod \"must-gather-9m2qj\" (UID: \"0a54de14-beb0-4721-a638-2d15eb4a52a1\") " pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.726641 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.726552 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0a54de14-beb0-4721-a638-2d15eb4a52a1-must-gather-output\") pod \"must-gather-9m2qj\" (UID: \"0a54de14-beb0-4721-a638-2d15eb4a52a1\") " pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.726813 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.726798 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0a54de14-beb0-4721-a638-2d15eb4a52a1-must-gather-output\") pod \"must-gather-9m2qj\" (UID: \"0a54de14-beb0-4721-a638-2d15eb4a52a1\") " pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.734240 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.734219 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7rfx\" (UniqueName: \"kubernetes.io/projected/0a54de14-beb0-4721-a638-2d15eb4a52a1-kube-api-access-l7rfx\") pod \"must-gather-9m2qj\" (UID: \"0a54de14-beb0-4721-a638-2d15eb4a52a1\") " pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.816715 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.816654 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvj9/must-gather-9m2qj" Jul 12 15:21:52.945049 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.945001 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvj9/must-gather-9m2qj"] Jul 12 15:21:52.946688 ip-10-0-142-78 kubenswrapper[2573]: W0712 15:21:52.946662 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a54de14_beb0_4721_a638_2d15eb4a52a1.slice/crio-13ffe6b4c2f92da4798f61898aa4045a288831f0a586c6451e26826cedfe15c8 WatchSource:0}: Error finding container 13ffe6b4c2f92da4798f61898aa4045a288831f0a586c6451e26826cedfe15c8: Status 404 returned error can't find the container with id 13ffe6b4c2f92da4798f61898aa4045a288831f0a586c6451e26826cedfe15c8 Jul 12 15:21:52.948491 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:52.948476 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:21:53.323224 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:53.323184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvj9/must-gather-9m2qj" event={"ID":"0a54de14-beb0-4721-a638-2d15eb4a52a1","Type":"ContainerStarted","Data":"13ffe6b4c2f92da4798f61898aa4045a288831f0a586c6451e26826cedfe15c8"} Jul 12 15:21:54.329911 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:54.329876 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvj9/must-gather-9m2qj" event={"ID":"0a54de14-beb0-4721-a638-2d15eb4a52a1","Type":"ContainerStarted","Data":"28997b28e137b813bdd7f7fbec59a4efcc8107594d95a49041d19b5688e90424"} Jul 12 15:21:54.330333 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:54.329921 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvj9/must-gather-9m2qj" event={"ID":"0a54de14-beb0-4721-a638-2d15eb4a52a1","Type":"ContainerStarted","Data":"f7ec1c21981847e0065c69127738ec314cb730c70d90c311c4873b1603b0c92d"} Jul 12 15:21:54.345039 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:54.344949 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqvj9/must-gather-9m2qj" podStartSLOduration=1.433126347 podStartE2EDuration="2.344928877s" totalImagesPullingTime="911.80253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:54.342724014 +0000 UTC m=+2462.310607267" watchObservedRunningTime="2026-07-12 15:21:54.344928877 +0000 UTC m=+2462.312812130" Jul 12 15:21:55.383055 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:55.382992 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jbkhl_ee45648f-77ef-4944-aa18-e1c690675ef4/global-pull-secret-syncer/0.log" Jul 12 15:21:55.464376 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:55.464350 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2pmbg_a63be01d-18d1-40e1-8864-fb315b971072/konnectivity-agent/0.log" Jul 12 15:21:55.683690 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:21:55.683607 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-78.ec2.internal_e42d2ac267c25452ab1b32df9632db77/haproxy/0.log" Jul 12 15:22:00.039040 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.038985 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7_8507da11-9656-4b5f-9f36-105cb2fdae9a/extract/0.log" Jul 12 15:22:00.061945 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.061913 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7_8507da11-9656-4b5f-9f36-105cb2fdae9a/util/0.log" Jul 12 15:22:00.084582 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.084530 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lp9b7_8507da11-9656-4b5f-9f36-105cb2fdae9a/pull/0.log" Jul 12 15:22:00.111042 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.110991 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb_309ebb07-a049-4716-a791-e88f343d62d9/extract/0.log" Jul 12 15:22:00.131969 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.131937 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb_309ebb07-a049-4716-a791-e88f343d62d9/util/0.log" Jul 12 15:22:00.152158 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.152125 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e026lvb_309ebb07-a049-4716-a791-e88f343d62d9/pull/0.log" Jul 12 15:22:00.175524 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.175490 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49_f48a30dc-8e56-4d30-a219-75661726d3c5/extract/0.log" Jul 12 15:22:00.195247 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.195217 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49_f48a30dc-8e56-4d30-a219-75661726d3c5/util/0.log" Jul 12 15:22:00.215777 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.215729 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g2s49_f48a30dc-8e56-4d30-a219-75661726d3c5/pull/0.log" Jul 12 15:22:00.241562 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.241532 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7_cf2e9876-c6ef-4356-a884-14ec05b98a45/extract/0.log" Jul 12 15:22:00.261438 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.261407 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7_cf2e9876-c6ef-4356-a884-14ec05b98a45/util/0.log" Jul 12 15:22:00.280632 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.280598 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gczh7_cf2e9876-c6ef-4356-a884-14ec05b98a45/pull/0.log" Jul 12 15:22:00.637033 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:00.636662 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84dcbdc54b-scqgq_a1442e75-527f-4a31-bc6d-ea14bf4f6a13/authorino/0.log" Jul 12 15:22:03.141476 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.141425 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lpggh_e433612d-2ea8-4494-bb5e-92e37affccbb/node-exporter/0.log" Jul 12 15:22:03.165041 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.164974 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lpggh_e433612d-2ea8-4494-bb5e-92e37affccbb/kube-rbac-proxy/0.log" Jul 12 15:22:03.190180 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.190150 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lpggh_e433612d-2ea8-4494-bb5e-92e37affccbb/init-textfile/0.log" Jul 12 15:22:03.593505 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.593420 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-cxntn_e381d1ab-46c6-4822-9f57-e80d038ed4bf/prometheus-operator/0.log" Jul 12 15:22:03.610728 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.610702 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-cxntn_e381d1ab-46c6-4822-9f57-e80d038ed4bf/kube-rbac-proxy/0.log" Jul 12 15:22:03.668472 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.668442 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b96bf6f66-2kxjz_25c71d36-7402-4dbd-b893-f4524c9d7361/telemeter-client/0.log" Jul 12 15:22:03.689712 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.689682 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b96bf6f66-2kxjz_25c71d36-7402-4dbd-b893-f4524c9d7361/reload/0.log" Jul 12 15:22:03.708873 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:03.708847 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b96bf6f66-2kxjz_25c71d36-7402-4dbd-b893-f4524c9d7361/kube-rbac-proxy/0.log" Jul 12 15:22:04.938567 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:04.938525 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-pstf2_10e3433a-1692-4ea9-8e91-be8d0739b2d6/networking-console-plugin/0.log" Jul 12 15:22:05.095569 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.095532 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd"] Jul 12 15:22:05.103263 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.103230 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.109144 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.109104 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd"] Jul 12 15:22:05.173486 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.173446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-podres\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.173789 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.173763 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c98jw\" (UniqueName: \"kubernetes.io/projected/d4d397d3-1770-4357-b0c8-937bc428c7e8-kube-api-access-c98jw\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.173982 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.173966 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-lib-modules\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.174171 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.174153 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-proc\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.174382 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.174363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-sys\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.275659 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.275565 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-lib-modules\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.275659 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.275627 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-proc\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.275905 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.275711 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-sys\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.275905 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.275770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-podres\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.275905 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.275800 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c98jw\" (UniqueName: \"kubernetes.io/projected/d4d397d3-1770-4357-b0c8-937bc428c7e8-kube-api-access-c98jw\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.276340 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.276211 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-sys\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.276340 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.276279 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-lib-modules\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.276340 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.276279 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-podres\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.276610 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.276589 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d4d397d3-1770-4357-b0c8-937bc428c7e8-proc\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.285121 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.285095 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c98jw\" (UniqueName: \"kubernetes.io/projected/d4d397d3-1770-4357-b0c8-937bc428c7e8-kube-api-access-c98jw\") pod \"perf-node-gather-daemonset-9t6zd\" (UID: \"d4d397d3-1770-4357-b0c8-937bc428c7e8\") " pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.417141 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.417104 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:05.559447 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.559328 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/1.log" Jul 12 15:22:05.565824 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.565792 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gg4r2_6a83e5a4-34bc-4272-b2b5-5efe836a0ef2/console-operator/2.log" Jul 12 15:22:05.579317 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:05.579207 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd"] Jul 12 15:22:06.443709 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:06.443666 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" event={"ID":"d4d397d3-1770-4357-b0c8-937bc428c7e8","Type":"ContainerStarted","Data":"7e6dc62ed3f85da2620e06220ca0ea85731635787c72d55fe5466e7f7276bad6"} Jul 12 15:22:06.444157 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:06.443749 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" event={"ID":"d4d397d3-1770-4357-b0c8-937bc428c7e8","Type":"ContainerStarted","Data":"b19833e9bcfd1e7626a20fe9a21cfb05f5e43de7ab7d0903231d8750f6d95295"} Jul 12 15:22:06.444157 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:06.443859 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:06.460101 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:06.460052 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" podStartSLOduration=1.46003589 podStartE2EDuration="1.46003589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:22:06.456985664 +0000 UTC m=+2474.424868916" watchObservedRunningTime="2026-07-12 15:22:06.46003589 +0000 UTC m=+2474.427919141" Jul 12 15:22:07.732506 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:07.732474 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rjb5_d1e45d8b-95d2-4193-96df-9749957d03a7/dns/0.log" Jul 12 15:22:07.753145 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:07.753117 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rjb5_d1e45d8b-95d2-4193-96df-9749957d03a7/kube-rbac-proxy/0.log" Jul 12 15:22:07.905542 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:07.905516 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ppcrh_7aa6132f-7dbf-439b-8683-2d1aca184a98/dns-node-resolver/0.log" Jul 12 15:22:08.470286 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:08.470255 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6mbxr_3043ff23-e0c3-4c6a-80a0-80566bc79d76/node-ca/0.log" Jul 12 15:22:09.376318 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:09.375772 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t6vpt_ee867255-1dd5-465b-81ab-e7f2d4b5d5cb/istio-proxy/0.log" Jul 12 15:22:09.489560 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:09.489513 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b855bfc69-bdqdf_5075b719-f93f-4559-beda-a6c11a335af9/kube-auth-proxy/0.log" Jul 12 15:22:10.367804 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:10.367766 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ckf4h_b4aa18e4-0242-415b-90bc-af206b09567f/serve-healthcheck-canary/0.log" Jul 12 15:22:10.886915 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:10.886889 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-br9f6_718e87e6-ce7d-4af0-ba12-b9783102d8aa/kube-rbac-proxy/0.log" Jul 12 15:22:10.907086 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:10.907058 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-br9f6_718e87e6-ce7d-4af0-ba12-b9783102d8aa/exporter/0.log" Jul 12 15:22:10.926087 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:10.926057 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-br9f6_718e87e6-ce7d-4af0-ba12-b9783102d8aa/extractor/0.log" Jul 12 15:22:12.464232 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:12.464199 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pqvj9/perf-node-gather-daemonset-9t6zd" Jul 12 15:22:13.438308 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:13.438281 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-56fs9_65758d6f-63fb-4b2a-b915-6fa74894d187/server/0.log" Jul 12 15:22:13.614872 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:13.614809 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9f5575-88csv_9bb28d76-211c-44e3-9651-07b03bbde796/manager/0.log" Jul 12 15:22:13.630547 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:13.630501 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d4d9f5575-88csv_9bb28d76-211c-44e3-9651-07b03bbde796/copy-manifests/0.log" Jul 12 15:22:15.095806 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:15.095751 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f9f5d4d9d-2mszd_af94111a-b8e8-4543-b6b2-b65f73191f24/manager/0.log" Jul 12 15:22:15.106710 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:15.106680 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t6vpt_ee867255-1dd5-465b-81ab-e7f2d4b5d5cb/istio-proxy/0.log" Jul 12 15:22:15.151907 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:15.151875 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pfvt2_b94d9afd-0f1e-4b1d-b501-6e74981f1090/openshift-lws-operator/0.log" Jul 12 15:22:15.214497 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:15.214421 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b855bfc69-bdqdf_5075b719-f93f-4559-beda-a6c11a335af9/kube-auth-proxy/0.log" Jul 12 15:22:16.560111 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:16.560070 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t6vpt_ee867255-1dd5-465b-81ab-e7f2d4b5d5cb/istio-proxy/0.log" Jul 12 15:22:16.661129 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:16.661098 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b855bfc69-bdqdf_5075b719-f93f-4559-beda-a6c11a335af9/kube-auth-proxy/0.log" Jul 12 15:22:20.241531 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:20.241502 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-n6vd5_4d3fd18c-856a-42ec-a45c-2087f975bd1f/kube-storage-version-migrator-operator/1.log" Jul 12 15:22:20.242339 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:20.242302 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-n6vd5_4d3fd18c-856a-42ec-a45c-2087f975bd1f/kube-storage-version-migrator-operator/0.log" Jul 12 15:22:21.183181 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.183151 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-567hq_0e85603c-a47b-4425-b415-f66384553038/kube-multus-additional-cni-plugins/0.log" Jul 12 15:22:21.202518 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.202492 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-567hq_0e85603c-a47b-4425-b415-f66384553038/egress-router-binary-copy/0.log" Jul 12 15:22:21.221571 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.221547 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-567hq_0e85603c-a47b-4425-b415-f66384553038/cni-plugins/0.log" Jul 12 15:22:21.240833 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.240813 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-567hq_0e85603c-a47b-4425-b415-f66384553038/bond-cni-plugin/0.log" Jul 12 15:22:21.262107 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.262088 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-567hq_0e85603c-a47b-4425-b415-f66384553038/routeoverride-cni/0.log" Jul 12 15:22:21.281685 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.281662 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-567hq_0e85603c-a47b-4425-b415-f66384553038/whereabouts-cni-bincopy/0.log" Jul 12 15:22:21.300200 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.300178 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-567hq_0e85603c-a47b-4425-b415-f66384553038/whereabouts-cni/0.log" Jul 12 15:22:21.685004 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.684979 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rkmbl_903d8cfd-d90e-42ca-8a0e-54b2763363ac/kube-multus/0.log" Jul 12 15:22:21.748060 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.748009 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fjw4t_9ab38523-f4fb-4fb3-9111-4dd8070e71de/network-metrics-daemon/0.log" Jul 12 15:22:21.765125 ip-10-0-142-78 kubenswrapper[2573]: I0712 15:22:21.765102 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fjw4t_9ab38523-f4fb-4fb3-9111-4dd8070e71de/kube-rbac-proxy/0.log"