Jul 08 18:02:11.812834 ip-10-0-141-56 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 18:02:11.812846 ip-10-0-141-56 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 18:02:11.812856 ip-10-0-141-56 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 18:02:11.813169 ip-10-0-141-56 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 18:02:22.059114 ip-10-0-141-56 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 18:02:22.059131 ip-10-0-141-56 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3ac58b5c6d52474e80ff25a200b47235 -- Jul 08 18:04:04.207149 ip-10-0-141-56 systemd[1]: Starting Kubernetes Kubelet... Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.611865 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616767 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616798 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616804 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616808 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616812 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616816 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616819 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:04.618386 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616823 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616826 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616845 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616848 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616852 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616855 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616858 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616862 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616866 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616869 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616872 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616876 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616880 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616884 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616887 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616890 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616894 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616897 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616902 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:04.632862 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616905 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616908 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616912 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616917 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616922 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616926 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616930 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616933 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616937 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616940 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616944 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616949 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616953 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616956 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616960 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616963 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616968 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616972 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616976 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616980 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:04.634645 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616984 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616990 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.616995 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617000 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617004 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617008 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617014 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617018 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617022 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617026 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617031 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617037 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617041 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617047 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617051 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617056 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617060 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617065 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617069 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617074 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:04.637048 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617077 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617081 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617085 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617089 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617094 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617108 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617112 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617116 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617120 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617124 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617128 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617132 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617136 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617140 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617146 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617151 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617156 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617160 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617164 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:04.648520 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617168 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617172 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617179 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617182 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617188 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617192 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617196 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617200 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617203 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617207 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617211 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617215 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617219 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.617223 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618054 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618061 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618065 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618068 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618072 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618077 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:04.650770 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618080 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618083 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618086 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618090 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618093 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618096 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618099 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618102 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618104 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618107 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618109 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618112 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618114 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618117 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618121 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618123 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618126 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618129 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618132 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618135 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618137 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:04.652686 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618140 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618142 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618145 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618147 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618150 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618152 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618155 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618158 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618160 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618163 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618165 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618168 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618170 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618173 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618176 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618179 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618181 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618183 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618186 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618188 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:04.655022 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618191 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618193 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618197 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618201 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618204 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618208 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618211 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618214 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618217 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618220 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618223 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618225 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618228 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618230 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618233 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618235 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618238 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618240 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618243 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618245 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:04.664540 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618248 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618250 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618253 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618255 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618259 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618261 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618265 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618268 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618271 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618275 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618278 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618282 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618286 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618289 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618294 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618298 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618301 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618307 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618311 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:04.667099 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618315 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618322 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618327 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618332 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618336 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618340 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618346 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618351 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618355 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618360 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618364 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618368 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618373 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618939 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618949 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618956 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618961 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618974 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618979 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:04.668873 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618984 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618989 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618993 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.618997 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619001 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619006 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619010 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619014 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619018 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619022 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619026 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619033 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619038 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619042 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619047 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619051 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619055 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619059 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619063 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619068 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:04.671148 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619072 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619076 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619082 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619087 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619091 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619095 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619099 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619103 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619131 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619137 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619143 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619160 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619165 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619169 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619173 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619178 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619182 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619186 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619190 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619194 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:04.753061 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619198 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:04.697925 ip-10-0-141-56 systemd[1]: Started Kubernetes Kubelet. Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619203 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619208 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619214 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619218 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619223 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619227 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619231 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619235 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619239 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619243 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619247 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619251 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619256 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619262 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619266 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619271 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619275 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619279 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619283 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:04.931625 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619287 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619291 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619295 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619300 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619311 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619316 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619320 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619324 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619330 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619337 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619343 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619348 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619353 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619358 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619362 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619368 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619373 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619377 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:04.944261 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619382 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619386 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619390 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619394 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619398 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619402 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619407 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619411 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619425 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619429 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619433 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619437 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619441 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619445 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.619449 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619566 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619580 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619590 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619598 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619605 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619610 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 18:04:04.945313 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619617 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619630 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619635 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619640 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619645 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619649 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619654 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619658 2580 flags.go:64] FLAG: --cgroup-root="" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619665 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619669 2580 flags.go:64] FLAG: --client-ca-file="" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619674 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619678 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619684 2580 flags.go:64] FLAG: --cluster-domain="" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619688 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619692 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619697 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619702 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619707 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619712 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619719 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619724 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619728 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619732 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619736 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 18:04:04.946145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619741 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619745 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619751 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619756 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619760 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619764 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619769 2580 flags.go:64] FLAG: --enable-server="true" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619774 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619780 2580 flags.go:64] FLAG: --event-burst="100" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619784 2580 flags.go:64] FLAG: --event-qps="50" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619788 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619793 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619797 2580 flags.go:64] FLAG: --eviction-hard="" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619803 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619807 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619812 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619818 2580 flags.go:64] FLAG: --eviction-soft="" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619823 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619827 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619849 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619854 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619857 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619862 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619866 2580 flags.go:64] FLAG: --feature-gates="" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619871 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 18:04:04.947185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619876 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619880 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619886 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619890 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619895 2580 flags.go:64] FLAG: --help="false" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619900 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-141-56.ec2.internal" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619905 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619909 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619912 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619915 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619919 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619922 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619925 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619927 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619930 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619934 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619937 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619940 2580 flags.go:64] FLAG: --kube-reserved="" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619943 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619946 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619949 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619952 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619955 2580 flags.go:64] FLAG: --lock-file="" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619959 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 18:04:04.948034 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619962 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619965 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619971 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619974 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619976 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619980 2580 flags.go:64] FLAG: --logging-format="text" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619982 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619985 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619988 2580 flags.go:64] FLAG: --manifest-url="" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619991 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.619998 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620001 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620005 2580 flags.go:64] FLAG: --max-pods="110" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620008 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620011 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620014 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620017 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620020 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620023 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620025 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620031 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620034 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620037 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 18:04:04.948889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620040 2580 flags.go:64] FLAG: --pod-cidr="" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620044 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620049 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620052 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620055 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620058 2580 flags.go:64] FLAG: --port="10250" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620061 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620064 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c0f0d39cd9dd4c4a" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620067 2580 flags.go:64] FLAG: --qos-reserved="" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620071 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620074 2580 flags.go:64] FLAG: --register-node="true" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620077 2580 flags.go:64] FLAG: --register-with-taints="" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620081 2580 flags.go:64] FLAG: --registry-burst="10" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620084 2580 flags.go:64] FLAG: --registry-qps="5" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620086 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620089 2580 flags.go:64] FLAG: --reserved-memory="" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620093 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620096 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620099 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620103 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620106 2580 flags.go:64] FLAG: --runonce="false" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620109 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620112 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620115 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620118 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 18:04:04.949712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620120 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620123 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620126 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620129 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620132 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620135 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620138 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620140 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620143 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620146 2580 flags.go:64] FLAG: --system-cgroups="" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620150 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620155 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620158 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620160 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620164 2580 flags.go:64] FLAG: --tls-min-version="" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620167 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620171 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620173 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620176 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620179 2580 flags.go:64] FLAG: --v="2" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620183 2580 flags.go:64] FLAG: --version="false" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620187 2580 flags.go:64] FLAG: --vmodule="" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620190 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.620194 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 18:04:04.950570 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620318 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620322 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620327 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620329 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620332 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620335 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620338 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620340 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620343 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620345 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620348 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620351 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620353 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620356 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620358 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620361 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620364 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620367 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620375 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:04.951402 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620378 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620380 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620383 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620385 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620388 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620393 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620395 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620398 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620400 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620403 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620406 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620408 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620411 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620413 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620416 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620420 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620422 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620425 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620427 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620430 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:04.952040 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620432 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620435 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620437 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620440 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620442 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620445 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620448 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620450 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620453 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620455 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620459 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620463 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620472 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620475 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620477 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620480 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620483 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620489 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620492 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620494 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:04.952678 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620497 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620500 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620502 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620505 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620507 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620510 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620512 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620516 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620519 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620521 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620524 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620526 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620529 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620531 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620534 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620536 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620539 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620541 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620544 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:04.953282 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620548 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620552 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620556 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620559 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620562 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620565 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620569 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620571 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620573 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620576 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620580 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620583 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620585 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620588 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620591 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620593 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620596 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620598 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620601 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620603 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:04.954133 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.620607 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.621395 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.628359 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.628384 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.629294 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.629305 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629362 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629368 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629372 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629375 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629378 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629381 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629384 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629387 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629389 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:04.954756 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629392 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629395 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629398 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629401 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629404 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629407 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629410 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629413 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629416 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629419 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629422 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629424 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629427 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629430 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629433 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629435 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629438 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629440 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629443 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629445 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:04.955603 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629449 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629451 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629454 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629456 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629459 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629461 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629464 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629466 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629469 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629471 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629474 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629476 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629479 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629482 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629484 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629486 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629489 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629492 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629495 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629497 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:04.956265 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629500 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629502 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629505 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629507 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629510 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629513 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629516 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629519 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629521 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629524 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629526 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629529 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629531 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629535 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629537 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629539 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629542 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629545 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629547 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629554 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:04.956799 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629557 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629560 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629562 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629565 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629568 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629570 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629574 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629577 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629580 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629583 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629587 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629590 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629593 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629596 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629599 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629602 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629605 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629607 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629610 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:04.957439 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629613 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629616 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629618 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629622 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629630 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629632 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629635 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629638 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629643 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629646 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629649 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.629655 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629771 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629777 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:04.957981 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629780 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629783 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629786 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629789 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629791 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629796 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629798 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629801 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629804 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629806 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629809 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629812 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629815 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629818 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629820 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629823 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629825 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629828 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629847 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629851 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:04.958420 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629853 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629856 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629859 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629862 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629864 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629867 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629871 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629873 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629876 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629878 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629881 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629885 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629887 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629890 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629892 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629895 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629898 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629902 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629904 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629907 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:04.959266 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629910 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629912 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629915 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629919 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629922 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629925 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629928 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629931 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629934 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629937 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629940 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629942 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629945 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629947 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629950 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629952 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629955 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629957 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629960 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:05.050954 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629968 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629970 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629973 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629975 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629979 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629982 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629985 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629987 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629990 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629992 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629996 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.629999 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630001 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630004 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630007 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630009 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630012 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630014 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630017 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630020 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:05.054084 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630023 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630025 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630028 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630031 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630034 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630036 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630039 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630041 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630044 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630047 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630050 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630053 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630055 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630059 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630061 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630064 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630068 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:05.054730 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:04.630070 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.630075 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.630188 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.633920 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.635693 2580 server.go:1019] "Starting client certificate rotation" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.635800 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.636647 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.638443 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.644737 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.658701 2580 log.go:25] "Validated CRI v1 runtime API" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.661236 2580 log.go:25] "Validated CRI v1 image API" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.661970 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 18:04:05.055207 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.664459 2580 fs.go:135] Filesystem UUIDs: map[3d32ecc0-3a76-434f-ac9a-4c90620a3c9d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 91a47679-df21-4564-9d06-11f22593bc27:/dev/nvme0n1p4] Jul 08 18:04:05.055601 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.664489 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 18:04:05.055670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.670230 2580 manager.go:216] Machine: {Timestamp:2026-07-08 18:04:04.668799685 +0000 UTC m=+0.354006444 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100317 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28618e008014f4fc183065b67aeb4e SystemUUID:ec28618e-0080-14f4-fc18-3065b67aeb4e BootID:3ac58b5c-6d52-474e-80ff-25a200b47235 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:15:7a:18:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:15:7a:18:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:7f:d0:cf:12:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 18:04:05.055670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.670376 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 18:04:05.055670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.670449 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 18:04:05.055670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.672402 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:04:05.055670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.674148 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 18:04:05.055670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.674190 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-56.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 18:04:05.055670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.675072 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.675083 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.675091 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.675881 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.677750 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.682418 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.682537 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.682567 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.682579 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.682596 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.682608 2580 kubelet.go:398] "Adding apiserver pod source" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.682617 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.683004 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.683019 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.687065 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.690057 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.690085 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692081 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692101 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692110 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692119 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692131 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692139 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692151 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692159 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692170 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692178 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692189 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.692198 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.693959 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.693970 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.694500 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-56.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.694505 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.697207 2580 server.go:1268] "Started kubelet" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.697284 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.697359 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.697308 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.698639 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.699311 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.703089 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 18:04:05.055977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.707424 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-56.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.707398 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-56.ec2.internal.18c062950a0d4fa2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-56.ec2.internal,UID:ip-10-0-141-56.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-56.ec2.internal,},FirstTimestamp:2026-07-08 18:04:04.697010082 +0000 UTC m=+0.382216842,LastTimestamp:2026-07-08 18:04:04.697010082 +0000 UTC m=+0.382216842,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-56.ec2.internal,}" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710023 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710043 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.710133 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710777 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710780 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710822 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710912 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710925 2580 factory.go:55] Registering systemd factory Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710934 2580 factory.go:223] Registration of the systemd container factory successfully Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710951 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.710958 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.711132 2580 factory.go:153] Registering CRI-O factory Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.711144 2580 factory.go:223] Registration of the crio container factory successfully Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.711160 2580 factory.go:103] Registering Raw factory Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.711170 2580 manager.go:1196] Started watching for new ooms in manager Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.711624 2580 manager.go:349] Starting recovery of all containers Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.711896 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.724061 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-56.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.724848 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-624jc" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.726721 2580 manager.go:354] Recovery completed Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.731068 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.732942 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.732974 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.732988 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.733180 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-624jc" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.733613 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.733621 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.733634 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.733651 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.734330 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:05.057016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.734356 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.734369 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.736742 2580 policy_none.go:49] "None policy: Start" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.736754 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.736764 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.739184 2580 policy_none.go:47] "Start" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.785441 2580 manager.go:337] "Starting Device Plugin manager" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.785475 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.785488 2580 server.go:87] "Starting device plugin registration server" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.785734 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.785746 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.785867 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.785971 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.785980 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.786368 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.786412 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.861555 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.862731 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.862766 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.862785 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.862816 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.865522 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.886617 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.888104 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.888143 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.888155 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.888175 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.898265 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.898282 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-56.ec2.internal\": node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:04.924037 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.963350 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal"] Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.963549 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.964493 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.964524 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.964535 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.966975 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.967104 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.967131 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:05.057908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.967788 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.967818 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.967850 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.968315 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.968331 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.968340 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.970165 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.970207 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.986859 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.986891 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:04.986904 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.001463 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-56.ec2.internal\" not found" node="ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.014435 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-56.ec2.internal\" not found" node="ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.058880 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.024257 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.195497 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.115186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/836915b6922e9a09409b13134d65a463-config\") pod \"kube-apiserver-proxy-ip-10-0-141-56.ec2.internal\" (UID: \"836915b6922e9a09409b13134d65a463\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.195497 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.115228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be1cf8d52b21daf7b1cf1506ba0db3cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal\" (UID: \"be1cf8d52b21daf7b1cf1506ba0db3cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.195497 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.115249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be1cf8d52b21daf7b1cf1506ba0db3cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal\" (UID: \"be1cf8d52b21daf7b1cf1506ba0db3cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.195497 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.125228 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.215934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be1cf8d52b21daf7b1cf1506ba0db3cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal\" (UID: \"be1cf8d52b21daf7b1cf1506ba0db3cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.215961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be1cf8d52b21daf7b1cf1506ba0db3cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal\" (UID: \"be1cf8d52b21daf7b1cf1506ba0db3cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.215979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/836915b6922e9a09409b13134d65a463-config\") pod \"kube-apiserver-proxy-ip-10-0-141-56.ec2.internal\" (UID: \"836915b6922e9a09409b13134d65a463\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.216025 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/836915b6922e9a09409b13134d65a463-config\") pod \"kube-apiserver-proxy-ip-10-0-141-56.ec2.internal\" (UID: \"836915b6922e9a09409b13134d65a463\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.216040 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be1cf8d52b21daf7b1cf1506ba0db3cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal\" (UID: \"be1cf8d52b21daf7b1cf1506ba0db3cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.216049 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be1cf8d52b21daf7b1cf1506ba0db3cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal\" (UID: \"be1cf8d52b21daf7b1cf1506ba0db3cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.226037 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.315084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.305244 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.414067 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.317621 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" Jul 08 18:04:05.414067 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.326136 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.505534 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.426681 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.612333 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.527123 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.716741 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.628308 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.716741 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.634965 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 18:04:05.716741 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.635098 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:04:05.824746 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.728630 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:05.824746 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.738986 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 17:59:05 +0000 UTC" deadline="2028-03-19 13:31:10.840262446 +0000 UTC" Jul 08 18:04:05.824746 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.739016 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14875h27m5.10125063s" Jul 08 18:04:05.824746 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:05.750383 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:05.829501 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.829163 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:06.003301 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:05.929231 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:06.080299 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.029642 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:06.163459 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.130272 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:06.294392 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.231211 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-56.ec2.internal\" not found" Jul 08 18:04:06.294392 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.246595 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:06.297677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.297142 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:06.301091 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:06.300484 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836915b6922e9a09409b13134d65a463.slice/crio-f0bfc424144a68e3402171f002294011ee323899dd53a04c6728cf27db02858e WatchSource:0}: Error finding container f0bfc424144a68e3402171f002294011ee323899dd53a04c6728cf27db02858e: Status 404 returned error can't find the container with id f0bfc424144a68e3402171f002294011ee323899dd53a04c6728cf27db02858e Jul 08 18:04:06.426869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.309192 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:04:06.426869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.310216 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" Jul 08 18:04:06.426869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.321073 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:04:06.426869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.322007 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" Jul 08 18:04:06.426869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.335728 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:04:06.686971 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.686955 2580 apiserver.go:52] "Watching apiserver" Jul 08 18:04:06.697341 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.697322 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 18:04:06.697744 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.697722 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-znlq7","kube-system/konnectivity-agent-9hvmv","openshift-cluster-node-tuning-operator/tuned-68sht","openshift-dns/node-resolver-lngwj","openshift-multus/multus-additional-cni-plugins-46qjq","openshift-multus/multus-tbfx6","openshift-multus/network-metrics-daemon-xvmqm","openshift-network-diagnostics/network-check-target-gkkkh","kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt","openshift-image-registry/node-ca-5tw7m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal","openshift-network-operator/iptables-alerter-t89mc","openshift-ovn-kubernetes/ovnkube-node-dpdlj"] Jul 08 18:04:06.701828 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.701811 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.704093 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.704067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:06.704450 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.704434 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 18:04:06.704615 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.704599 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7w4fd\"" Jul 08 18:04:06.704752 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.704737 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 18:04:06.704812 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.704804 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 18:04:06.708848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.706564 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.708848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.708090 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 18:04:06.708848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.708377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 18:04:06.708848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.708646 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-42pt6\"" Jul 08 18:04:06.709183 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.709160 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:06.709346 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.709325 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:06.711213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.710446 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fsr2l\"" Jul 08 18:04:06.711213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.710550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.711213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.710703 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 18:04:06.712845 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.712825 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.715664 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.715647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.717890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.717877 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:06.720777 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.720756 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:06.723135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.723118 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.725168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.724934 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 18:04:06.725168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.724993 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 18:04:06.725168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725016 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 18:04:06.725168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725019 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bxd82\"" Jul 08 18:04:06.725168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725098 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 18:04:06.725168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725169 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 18:04:06.725516 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725191 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 18:04:06.725516 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725372 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9cst8\"" Jul 08 18:04:06.725516 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725455 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 18:04:06.725516 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725511 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 18:04:06.725685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725517 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:06.725685 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.725590 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:06.725685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725532 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ggjgj\"" Jul 08 18:04:06.725853 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725814 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 18:04:06.725853 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725828 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:06.726040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.725913 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:06.726517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-var-lib-kubelet\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.726556 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726541 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ba1141-ba92-4c0b-8dd3-94169bb69f78-tmp\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.726596 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.726634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-device-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.726679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/82f9ee10-112d-4d87-91b3-9619e72c1c03-konnectivity-ca\") pod \"konnectivity-agent-9hvmv\" (UID: \"82f9ee10-112d-4d87-91b3-9619e72c1c03\") " pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:06.726679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysctl-d\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.726785 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysctl-conf\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.726785 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-tuned\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.726893 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-os-release\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.726893 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.726893 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5cqg\" (UniqueName: \"kubernetes.io/projected/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-kube-api-access-b5cqg\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.726893 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-host\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.726893 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-tmp-dir\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.727116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726927 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-system-cni-dir\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.727116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cnibin\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.727116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.726977 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-kubernetes\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.727116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727002 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-socket-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.727116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727015 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nhwml\"" Jul 08 18:04:06.727116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-sys\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.727116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdsf4\" (UniqueName: \"kubernetes.io/projected/87ba1141-ba92-4c0b-8dd3-94169bb69f78-kube-api-access-gdsf4\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-sys-fs\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-lib-modules\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwtp9\" (UniqueName: \"kubernetes.io/projected/319f6f85-dcd0-4060-89f5-8c32c347ba0d-kube-api-access-xwtp9\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-registration-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysconfig\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727272 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-systemd\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.727357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-run\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.727638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s89j\" (UniqueName: \"kubernetes.io/projected/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-kube-api-access-2s89j\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.727638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727381 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.727638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.727638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-hosts-file\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.727638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cni-binary-copy\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.727638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/82f9ee10-112d-4d87-91b3-9619e72c1c03-agent-certs\") pod \"konnectivity-agent-9hvmv\" (UID: \"82f9ee10-112d-4d87-91b3-9619e72c1c03\") " pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:06.727638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.727489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-modprobe-d\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.728174 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.728158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.728261 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.728226 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:06.730618 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.730601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.733190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.733153 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.733673 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.733643 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:04:06.733769 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.733678 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 18:04:06.733769 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.733653 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 18:04:06.733913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.733808 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 18:04:06.734224 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.734207 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s7lt7\"" Jul 08 18:04:06.735478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.735457 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 18:04:06.735570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.735477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-522dx\"" Jul 08 18:04:06.735675 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.735654 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 18:04:06.736650 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.736635 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 18:04:06.736743 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.736726 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 18:04:06.736796 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.736762 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 18:04:06.739549 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.739532 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 18:04:06.753297 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.751471 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rzvxv" Jul 08 18:04:06.759938 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.759905 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rzvxv" Jul 08 18:04:06.812060 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.812035 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 18:04:06.827659 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827630 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-lib-modules\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.827764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwtp9\" (UniqueName: \"kubernetes.io/projected/319f6f85-dcd0-4060-89f5-8c32c347ba0d-kube-api-access-xwtp9\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.827764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827685 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-cni-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.827764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-run-ovn-kubernetes\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.827764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-registration-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.827764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-run\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s89j\" (UniqueName: \"kubernetes.io/projected/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-kube-api-access-2s89j\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827790 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827806 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827870 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7afcc208-15f0-4fc3-9c71-571edc687f52-iptables-alerter-script\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ee9becb-c169-4398-b46e-7daa536542a4-kubelet-config\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827919 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-lib-modules\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827941 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-k8s-cni-cncf-io\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827969 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.828029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.827993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-env-overrides\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.828345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828055 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/82f9ee10-112d-4d87-91b3-9619e72c1c03-agent-certs\") pod \"konnectivity-agent-9hvmv\" (UID: \"82f9ee10-112d-4d87-91b3-9619e72c1c03\") " pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:06.828345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-run\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.828345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828140 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-registration-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.828345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-var-lib-kubelet\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.828345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:06.828345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828320 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-var-lib-kubelet\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-log-socket\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828408 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovn-node-metrics-cert\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828424 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828440 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-kubelet\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/82f9ee10-112d-4d87-91b3-9619e72c1c03-konnectivity-ca\") pod \"konnectivity-agent-9hvmv\" (UID: \"82f9ee10-112d-4d87-91b3-9619e72c1c03\") " pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysctl-conf\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828524 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-slash\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828541 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-run-netns\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5cqg\" (UniqueName: \"kubernetes.io/projected/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-kube-api-access-b5cqg\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.828636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828620 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-host\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828670 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysctl-conf\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-tmp-dir\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828793 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-host\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828820 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-cni-bin\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828870 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-systemd\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-etc-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-cni-netd\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bb5t\" (UniqueName: \"kubernetes.io/projected/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-kube-api-access-4bb5t\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.828983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-kubernetes\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-cnibin\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829030 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-kubernetes\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829028 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/82f9ee10-112d-4d87-91b3-9619e72c1c03-konnectivity-ca\") pod \"konnectivity-agent-9hvmv\" (UID: \"82f9ee10-112d-4d87-91b3-9619e72c1c03\") " pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829050 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-hostroot\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.829158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829077 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-tmp-dir\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znbjw\" (UniqueName: \"kubernetes.io/projected/21ad1bba-365f-43c5-b06f-13af24af75c3-kube-api-access-znbjw\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ee9becb-c169-4398-b46e-7daa536542a4-dbus\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-socket-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdsf4\" (UniqueName: \"kubernetes.io/projected/87ba1141-ba92-4c0b-8dd3-94169bb69f78-kube-api-access-gdsf4\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829247 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-socket-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc8n2\" (UniqueName: \"kubernetes.io/projected/7afcc208-15f0-4fc3-9c71-571edc687f52-kube-api-access-sc8n2\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-system-cni-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-cni-binary-copy\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whx6p\" (UniqueName: \"kubernetes.io/projected/5e04054b-06b5-4b90-9891-3e66ab6942c0-kube-api-access-whx6p\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-node-log\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829338 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-cni-bin\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-sys-fs\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-socket-dir-parent\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.829695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-conf-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-kubelet\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829451 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-sys-fs\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovnkube-script-lib\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysconfig\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-systemd\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829582 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysconfig\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829588 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-netns\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-hosts-file\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cni-binary-copy\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829676 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-systemd\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829708 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-hosts-file\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829716 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-cni-multus\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.830305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc6kf\" (UniqueName: \"kubernetes.io/projected/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-kube-api-access-pc6kf\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-modprobe-d\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e04054b-06b5-4b90-9891-3e66ab6942c0-serviceca\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovnkube-config\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829902 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-modprobe-d\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ba1141-ba92-4c0b-8dd3-94169bb69f78-tmp\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.829994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-device-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830033 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-device-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysctl-d\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830082 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cni-binary-copy\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830068 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-tuned\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.830837 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-os-release\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7afcc208-15f0-4fc3-9c71-571edc687f52-host-slash\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e04054b-06b5-4b90-9891-3e66ab6942c0-host\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830195 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-sysctl-d\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830254 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-os-release\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-system-cni-dir\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830280 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cnibin\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830325 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-system-cni-dir\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830370 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-systemd-units\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830389 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-cnibin\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-ovn\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-os-release\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.831307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-multus-certs\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.831719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-etc-kubernetes\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.831719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-sys\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.831719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830580 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87ba1141-ba92-4c0b-8dd3-94169bb69f78-sys\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.831719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.831719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830640 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-daemon-config\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.831719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830672 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-var-lib-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.831719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.830777 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/319f6f85-dcd0-4060-89f5-8c32c347ba0d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.832243 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.832216 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87ba1141-ba92-4c0b-8dd3-94169bb69f78-etc-tuned\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.832371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.832353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87ba1141-ba92-4c0b-8dd3-94169bb69f78-tmp\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.832548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.832534 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/82f9ee10-112d-4d87-91b3-9619e72c1c03-agent-certs\") pod \"konnectivity-agent-9hvmv\" (UID: \"82f9ee10-112d-4d87-91b3-9619e72c1c03\") " pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:06.841982 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.841953 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5cqg\" (UniqueName: \"kubernetes.io/projected/a2561f6e-b1e8-47e9-8aca-14d832e8b3e2-kube-api-access-b5cqg\") pod \"aws-ebs-csi-driver-node-jrpqt\" (UID: \"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:06.842225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.842199 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s89j\" (UniqueName: \"kubernetes.io/projected/6d5114bf-8ccf-4e07-8295-8e0d447ca9e5-kube-api-access-2s89j\") pod \"node-resolver-lngwj\" (UID: \"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5\") " pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:06.842448 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.842432 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdsf4\" (UniqueName: \"kubernetes.io/projected/87ba1141-ba92-4c0b-8dd3-94169bb69f78-kube-api-access-gdsf4\") pod \"tuned-68sht\" (UID: \"87ba1141-ba92-4c0b-8dd3-94169bb69f78\") " pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:06.842501 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.842484 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwtp9\" (UniqueName: \"kubernetes.io/projected/319f6f85-dcd0-4060-89f5-8c32c347ba0d-kube-api-access-xwtp9\") pod \"multus-additional-cni-plugins-46qjq\" (UID: \"319f6f85-dcd0-4060-89f5-8c32c347ba0d\") " pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:06.867389 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.867342 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" event={"ID":"836915b6922e9a09409b13134d65a463","Type":"ContainerStarted","Data":"f0bfc424144a68e3402171f002294011ee323899dd53a04c6728cf27db02858e"} Jul 08 18:04:06.868432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.868413 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" event={"ID":"be1cf8d52b21daf7b1cf1506ba0db3cf","Type":"ContainerStarted","Data":"60e4b379ef208661c80fe464fa8e99798bee9e308fdf2c8891242e1f8bce2fd9"} Jul 08 18:04:06.893099 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.893077 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:06.931901 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.931803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-node-log\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.931901 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.931841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-cni-bin\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.931901 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.931881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-socket-dir-parent\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.932156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.931920 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-node-log\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.932156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932004 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-socket-dir-parent\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.932156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-conf-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.932156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932037 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-conf-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.932156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-kubelet\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.932156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.932365 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932172 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.932518 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.931927 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-cni-bin\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.932639 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932598 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-kubelet\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.932706 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovnkube-script-lib\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.932759 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-netns\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.932813 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-netns\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.932892 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932855 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-cni-multus\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933008 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.932989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-cni-multus\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933089 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc6kf\" (UniqueName: \"kubernetes.io/projected/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-kube-api-access-pc6kf\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933089 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:06.933186 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e04054b-06b5-4b90-9891-3e66ab6942c0-serviceca\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.933186 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovnkube-config\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.933186 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7afcc208-15f0-4fc3-9c71-571edc687f52-host-slash\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.933327 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e04054b-06b5-4b90-9891-3e66ab6942c0-host\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.933327 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-systemd-units\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.933327 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-ovn\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.933327 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-os-release\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933327 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-multus-certs\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-etc-kubernetes\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-daemon-config\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933408 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovnkube-script-lib\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.933548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933483 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-var-lib-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.933548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-var-lib-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.933548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-cni-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-run-ovn-kubernetes\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.933811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7afcc208-15f0-4fc3-9c71-571edc687f52-iptables-alerter-script\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.933811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ee9becb-c169-4398-b46e-7daa536542a4-kubelet-config\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.933811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933699 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-k8s-cni-cncf-io\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.933811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.933811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-env-overrides\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-log-socket\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovn-node-metrics-cert\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-kubelet\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-slash\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.933994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-run-netns\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934064 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7afcc208-15f0-4fc3-9c71-571edc687f52-host-slash\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.934116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-cni-bin\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934129 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e04054b-06b5-4b90-9891-3e66ab6942c0-host\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934144 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-cni-bin\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-systemd\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-etc-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-cni-netd\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934286 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-cni-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bb5t\" (UniqueName: \"kubernetes.io/projected/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-kube-api-access-4bb5t\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-cnibin\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-hostroot\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znbjw\" (UniqueName: \"kubernetes.io/projected/21ad1bba-365f-43c5-b06f-13af24af75c3-kube-api-access-znbjw\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ee9becb-c169-4398-b46e-7daa536542a4-dbus\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc8n2\" (UniqueName: \"kubernetes.io/projected/7afcc208-15f0-4fc3-9c71-571edc687f52-kube-api-access-sc8n2\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.934517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-system-cni-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-cni-binary-copy\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-systemd-units\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934571 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whx6p\" (UniqueName: \"kubernetes.io/projected/5e04054b-06b5-4b90-9891-3e66ab6942c0-kube-api-access-whx6p\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934689 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-ovn\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934770 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-os-release\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934829 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-multus-certs\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934878 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-etc-kubernetes\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.934878 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-run-ovn-kubernetes\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935004 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ee9becb-c169-4398-b46e-7daa536542a4-kubelet-config\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.935137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935056 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-run-k8s-cni-cncf-io\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.935598 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.935151 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:06.935598 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.935247 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret podName:7ee9becb-c169-4398-b46e-7daa536542a4 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:07.435203025 +0000 UTC m=+3.120409791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret") pod "global-pull-secret-syncer-znlq7" (UID: "7ee9becb-c169-4398-b46e-7daa536542a4") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:06.935598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935470 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-multus-daemon-config\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.935598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935531 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-run-systemd\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.935598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935586 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-etc-openvswitch\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.935832 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935644 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-cni-netd\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.935832 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935695 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.935832 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.935798 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:06.936033 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935837 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-env-overrides\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.936033 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.935859 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs podName:21ad1bba-365f-43c5-b06f-13af24af75c3 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:07.435839068 +0000 UTC m=+3.121045832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs") pod "network-metrics-daemon-xvmqm" (UID: "21ad1bba-365f-43c5-b06f-13af24af75c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:06.936033 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935943 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-log-socket\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.936193 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.935866 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-host-var-lib-kubelet\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.936330 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936307 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ee9becb-c169-4398-b46e-7daa536542a4-dbus\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:06.936418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936316 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-run-netns\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.936418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936387 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-cnibin\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.936418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-hostroot\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.936582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936447 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-cni-binary-copy\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.936582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936405 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovnkube-config\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.936582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936486 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e04054b-06b5-4b90-9891-3e66ab6942c0-serviceca\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:06.936582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936507 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7afcc208-15f0-4fc3-9c71-571edc687f52-iptables-alerter-script\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.936582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-host-slash\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.936760 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.936562 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-system-cni-dir\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.938986 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.938967 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-ovn-node-metrics-cert\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.944132 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.944115 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:06.944209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.944135 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:06.944209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.944146 2580 projected.go:196] Error preparing data for projected volume kube-api-access-79lgv for pod openshift-network-diagnostics/network-check-target-gkkkh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:06.944209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:06.944191 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv podName:19db2ce5-00a8-43fa-9927-c747b6fb9d9e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:07.444178615 +0000 UTC m=+3.129385387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-79lgv" (UniqueName: "kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv") pod "network-check-target-gkkkh" (UID: "19db2ce5-00a8-43fa-9927-c747b6fb9d9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:06.949708 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.949680 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bb5t\" (UniqueName: \"kubernetes.io/projected/e034a2a0-5921-4aeb-8e0a-9ca4f1994be5-kube-api-access-4bb5t\") pod \"ovnkube-node-dpdlj\" (UID: \"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5\") " pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:06.949780 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.949757 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc8n2\" (UniqueName: \"kubernetes.io/projected/7afcc208-15f0-4fc3-9c71-571edc687f52-kube-api-access-sc8n2\") pod \"iptables-alerter-t89mc\" (UID: \"7afcc208-15f0-4fc3-9c71-571edc687f52\") " pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:06.950138 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.950116 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znbjw\" (UniqueName: \"kubernetes.io/projected/21ad1bba-365f-43c5-b06f-13af24af75c3-kube-api-access-znbjw\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:06.950220 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.950151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc6kf\" (UniqueName: \"kubernetes.io/projected/0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd-kube-api-access-pc6kf\") pod \"multus-tbfx6\" (UID: \"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd\") " pod="openshift-multus/multus-tbfx6" Jul 08 18:04:06.950220 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:06.950162 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whx6p\" (UniqueName: \"kubernetes.io/projected/5e04054b-06b5-4b90-9891-3e66ab6942c0-kube-api-access-whx6p\") pod \"node-ca-5tw7m\" (UID: \"5e04054b-06b5-4b90-9891-3e66ab6942c0\") " pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:07.025729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.025693 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" Jul 08 18:04:07.032896 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.032870 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:07.036711 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.036680 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2561f6e_b1e8_47e9_8aca_14d832e8b3e2.slice/crio-3996ba19924ae07865630d5c55420c52e5937b53e21826b0c7a03fa45c3bc83b WatchSource:0}: Error finding container 3996ba19924ae07865630d5c55420c52e5937b53e21826b0c7a03fa45c3bc83b: Status 404 returned error can't find the container with id 3996ba19924ae07865630d5c55420c52e5937b53e21826b0c7a03fa45c3bc83b Jul 08 18:04:07.042781 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.042754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-68sht" Jul 08 18:04:07.043549 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.043523 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f9ee10_112d_4d87_91b3_9619e72c1c03.slice/crio-3c5d329009eb4440fa13ca649ff5cff634d682ca4a6c78689b3d37546dc8dc51 WatchSource:0}: Error finding container 3c5d329009eb4440fa13ca649ff5cff634d682ca4a6c78689b3d37546dc8dc51: Status 404 returned error can't find the container with id 3c5d329009eb4440fa13ca649ff5cff634d682ca4a6c78689b3d37546dc8dc51 Jul 08 18:04:07.048027 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.048004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lngwj" Jul 08 18:04:07.054312 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.054222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-46qjq" Jul 08 18:04:07.054528 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.054506 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ba1141_ba92_4c0b_8dd3_94169bb69f78.slice/crio-f6c7c4b48713a4faf947fd33a5e130954c9dac28299ab8eb6e09b9a007e972a6 WatchSource:0}: Error finding container f6c7c4b48713a4faf947fd33a5e130954c9dac28299ab8eb6e09b9a007e972a6: Status 404 returned error can't find the container with id f6c7c4b48713a4faf947fd33a5e130954c9dac28299ab8eb6e09b9a007e972a6 Jul 08 18:04:07.059248 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.059207 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5114bf_8ccf_4e07_8295_8e0d447ca9e5.slice/crio-68b9035d29e677ac9502cf2cfa0658362118f1c5220390e1647d09493ee11bd2 WatchSource:0}: Error finding container 68b9035d29e677ac9502cf2cfa0658362118f1c5220390e1647d09493ee11bd2: Status 404 returned error can't find the container with id 68b9035d29e677ac9502cf2cfa0658362118f1c5220390e1647d09493ee11bd2 Jul 08 18:04:07.063378 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.063353 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tbfx6" Jul 08 18:04:07.069695 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.069468 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319f6f85_dcd0_4060_89f5_8c32c347ba0d.slice/crio-ec24ba2a481916eebe2b4dc7a6fae075ad029bfbcbe5a431d98dcafeba05af0d WatchSource:0}: Error finding container ec24ba2a481916eebe2b4dc7a6fae075ad029bfbcbe5a431d98dcafeba05af0d: Status 404 returned error can't find the container with id ec24ba2a481916eebe2b4dc7a6fae075ad029bfbcbe5a431d98dcafeba05af0d Jul 08 18:04:07.069695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.069536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t89mc" Jul 08 18:04:07.075013 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.074871 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tw7m" Jul 08 18:04:07.075250 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.075225 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7ae6ce_0d7e_4ac8_8c64_a9f5292ed1cd.slice/crio-52f4cf23961c850275c913f8d856c039938b0a199ee5c235c60dba4d725826f4 WatchSource:0}: Error finding container 52f4cf23961c850275c913f8d856c039938b0a199ee5c235c60dba4d725826f4: Status 404 returned error can't find the container with id 52f4cf23961c850275c913f8d856c039938b0a199ee5c235c60dba4d725826f4 Jul 08 18:04:07.080849 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.080626 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:07.081348 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.081324 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7afcc208_15f0_4fc3_9c71_571edc687f52.slice/crio-5a73e7953877f2b945470fa91eab08e387fcf6e5ccb33b64392e73a81672e53b WatchSource:0}: Error finding container 5a73e7953877f2b945470fa91eab08e387fcf6e5ccb33b64392e73a81672e53b: Status 404 returned error can't find the container with id 5a73e7953877f2b945470fa91eab08e387fcf6e5ccb33b64392e73a81672e53b Jul 08 18:04:07.089305 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.089276 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e04054b_06b5_4b90_9891_3e66ab6942c0.slice/crio-472b4353778d34f1ba143cbfda928e869241889e591c43b8a74c0a9c6be8894a WatchSource:0}: Error finding container 472b4353778d34f1ba143cbfda928e869241889e591c43b8a74c0a9c6be8894a: Status 404 returned error can't find the container with id 472b4353778d34f1ba143cbfda928e869241889e591c43b8a74c0a9c6be8894a Jul 08 18:04:07.093016 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:07.092866 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode034a2a0_5921_4aeb_8e0a_9ca4f1994be5.slice/crio-a17a21cb14f12cc40ecf3a758168e7098565f16a87b88a88534c52c5613f2a79 WatchSource:0}: Error finding container a17a21cb14f12cc40ecf3a758168e7098565f16a87b88a88534c52c5613f2a79: Status 404 returned error can't find the container with id a17a21cb14f12cc40ecf3a758168e7098565f16a87b88a88534c52c5613f2a79 Jul 08 18:04:07.438746 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.438711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:07.438925 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.438764 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:07.438925 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.438894 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:07.439057 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.438958 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret podName:7ee9becb-c169-4398-b46e-7daa536542a4 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:08.438944031 +0000 UTC m=+4.124150780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret") pod "global-pull-secret-syncer-znlq7" (UID: "7ee9becb-c169-4398-b46e-7daa536542a4") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:07.439057 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.438896 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:07.439057 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.439032 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs podName:21ad1bba-365f-43c5-b06f-13af24af75c3 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:08.439019907 +0000 UTC m=+4.124226669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs") pod "network-metrics-daemon-xvmqm" (UID: "21ad1bba-365f-43c5-b06f-13af24af75c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:07.540030 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.539985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:07.540240 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.540207 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:07.540240 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.540235 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:07.540373 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.540250 2580 projected.go:196] Error preparing data for projected volume kube-api-access-79lgv for pod openshift-network-diagnostics/network-check-target-gkkkh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:07.540373 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.540317 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv podName:19db2ce5-00a8-43fa-9927-c747b6fb9d9e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:08.540298229 +0000 UTC m=+4.225504985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-79lgv" (UniqueName: "kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv") pod "network-check-target-gkkkh" (UID: "19db2ce5-00a8-43fa-9927-c747b6fb9d9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:07.763585 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.763480 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:59:07 +0000 UTC" deadline="2028-03-12 16:27:07.401949048 +0000 UTC" Jul 08 18:04:07.763585 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.763524 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14710h22m59.638428885s" Jul 08 18:04:07.864280 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.864247 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:07.864449 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.864400 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:07.864838 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.864813 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:07.864963 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:07.864906 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:07.876872 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.876785 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"a17a21cb14f12cc40ecf3a758168e7098565f16a87b88a88534c52c5613f2a79"} Jul 08 18:04:07.886120 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.886038 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t89mc" event={"ID":"7afcc208-15f0-4fc3-9c71-571edc687f52","Type":"ContainerStarted","Data":"5a73e7953877f2b945470fa91eab08e387fcf6e5ccb33b64392e73a81672e53b"} Jul 08 18:04:07.888390 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.888361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tbfx6" event={"ID":"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd","Type":"ContainerStarted","Data":"52f4cf23961c850275c913f8d856c039938b0a199ee5c235c60dba4d725826f4"} Jul 08 18:04:07.890558 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.890532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerStarted","Data":"ec24ba2a481916eebe2b4dc7a6fae075ad029bfbcbe5a431d98dcafeba05af0d"} Jul 08 18:04:07.896242 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.896216 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lngwj" event={"ID":"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5","Type":"ContainerStarted","Data":"68b9035d29e677ac9502cf2cfa0658362118f1c5220390e1647d09493ee11bd2"} Jul 08 18:04:07.899429 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.899405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9hvmv" event={"ID":"82f9ee10-112d-4d87-91b3-9619e72c1c03","Type":"ContainerStarted","Data":"3c5d329009eb4440fa13ca649ff5cff634d682ca4a6c78689b3d37546dc8dc51"} Jul 08 18:04:07.901158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.901132 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tw7m" event={"ID":"5e04054b-06b5-4b90-9891-3e66ab6942c0","Type":"ContainerStarted","Data":"472b4353778d34f1ba143cbfda928e869241889e591c43b8a74c0a9c6be8894a"} Jul 08 18:04:07.903442 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.903417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-68sht" event={"ID":"87ba1141-ba92-4c0b-8dd3-94169bb69f78","Type":"ContainerStarted","Data":"f6c7c4b48713a4faf947fd33a5e130954c9dac28299ab8eb6e09b9a007e972a6"} Jul 08 18:04:07.905772 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:07.905740 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" event={"ID":"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2","Type":"ContainerStarted","Data":"3996ba19924ae07865630d5c55420c52e5937b53e21826b0c7a03fa45c3bc83b"} Jul 08 18:04:08.447031 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:08.446988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:08.447031 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:08.447044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:08.447317 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.447193 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:08.447317 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.447288 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs podName:21ad1bba-365f-43c5-b06f-13af24af75c3 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:10.447265827 +0000 UTC m=+6.132472580 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs") pod "network-metrics-daemon-xvmqm" (UID: "21ad1bba-365f-43c5-b06f-13af24af75c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:08.447704 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.447578 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:08.447704 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.447669 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret podName:7ee9becb-c169-4398-b46e-7daa536542a4 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:10.447651523 +0000 UTC m=+6.132858283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret") pod "global-pull-secret-syncer-znlq7" (UID: "7ee9becb-c169-4398-b46e-7daa536542a4") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:08.548603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:08.547953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:08.548603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.548161 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:08.548603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.548184 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:08.548603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.548197 2580 projected.go:196] Error preparing data for projected volume kube-api-access-79lgv for pod openshift-network-diagnostics/network-check-target-gkkkh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:08.548603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.548257 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv podName:19db2ce5-00a8-43fa-9927-c747b6fb9d9e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:10.548238884 +0000 UTC m=+6.233445636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-79lgv" (UniqueName: "kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv") pod "network-check-target-gkkkh" (UID: "19db2ce5-00a8-43fa-9927-c747b6fb9d9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:08.764154 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:08.764036 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:59:07 +0000 UTC" deadline="2028-03-23 05:10:48.062310418 +0000 UTC" Jul 08 18:04:08.764154 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:08.764083 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14963h6m39.298233007s" Jul 08 18:04:08.864067 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:08.864036 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:08.864263 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:08.864204 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:09.863777 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:09.863733 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:09.864392 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:09.863904 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:09.864392 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:09.863963 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:09.864392 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:09.864092 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:10.464655 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:10.464616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:10.464876 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:10.464666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:10.464876 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.464785 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:10.464876 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.464852 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs podName:21ad1bba-365f-43c5-b06f-13af24af75c3 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:14.4648293 +0000 UTC m=+10.150036073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs") pod "network-metrics-daemon-xvmqm" (UID: "21ad1bba-365f-43c5-b06f-13af24af75c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:10.465080 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.464785 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:10.465080 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.464916 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret podName:7ee9becb-c169-4398-b46e-7daa536542a4 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:14.464906317 +0000 UTC m=+10.150113069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret") pod "global-pull-secret-syncer-znlq7" (UID: "7ee9becb-c169-4398-b46e-7daa536542a4") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:10.566125 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:10.566089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:10.566302 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.566263 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:10.566302 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.566292 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:10.566431 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.566308 2580 projected.go:196] Error preparing data for projected volume kube-api-access-79lgv for pod openshift-network-diagnostics/network-check-target-gkkkh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:10.566431 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.566377 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv podName:19db2ce5-00a8-43fa-9927-c747b6fb9d9e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:14.566356489 +0000 UTC m=+10.251563261 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-79lgv" (UniqueName: "kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv") pod "network-check-target-gkkkh" (UID: "19db2ce5-00a8-43fa-9927-c747b6fb9d9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:10.863787 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:10.863701 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:10.863979 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:10.863891 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:11.863551 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:11.863516 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:11.863746 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:11.863560 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:11.863746 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:11.863675 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:11.863906 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:11.863807 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:12.864680 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:12.864637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:12.865217 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:12.864785 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:13.863440 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:13.863353 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:13.863440 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:13.863396 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:13.863676 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:13.863522 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:13.863731 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:13.863665 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:14.503014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:14.502970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:14.503546 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:14.503030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:14.503546 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.503205 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:14.503546 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.503270 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs podName:21ad1bba-365f-43c5-b06f-13af24af75c3 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:22.503250667 +0000 UTC m=+18.188457431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs") pod "network-metrics-daemon-xvmqm" (UID: "21ad1bba-365f-43c5-b06f-13af24af75c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:14.503721 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.503688 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:14.503772 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.503737 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret podName:7ee9becb-c169-4398-b46e-7daa536542a4 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:22.503721902 +0000 UTC m=+18.188928653 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret") pod "global-pull-secret-syncer-znlq7" (UID: "7ee9becb-c169-4398-b46e-7daa536542a4") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:14.603980 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:14.603936 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:14.604156 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.604135 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:14.604246 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.604165 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:14.604246 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.604182 2580 projected.go:196] Error preparing data for projected volume kube-api-access-79lgv for pod openshift-network-diagnostics/network-check-target-gkkkh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:14.604356 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.604262 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv podName:19db2ce5-00a8-43fa-9927-c747b6fb9d9e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:22.604242845 +0000 UTC m=+18.289449612 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-79lgv" (UniqueName: "kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv") pod "network-check-target-gkkkh" (UID: "19db2ce5-00a8-43fa-9927-c747b6fb9d9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:14.867679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:14.867159 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:14.867679 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:14.867283 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:15.863014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:15.862976 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:15.863490 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:15.863116 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:15.863490 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:15.863162 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:15.863490 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:15.863287 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:16.863346 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:16.863302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:16.863791 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:16.863483 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:17.863720 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:17.863697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:17.864003 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:17.863697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:17.864003 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:17.863829 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:17.864003 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:17.863889 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:18.863511 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:18.863279 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:18.863682 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:18.863620 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:18.934764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:18.934726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-68sht" event={"ID":"87ba1141-ba92-4c0b-8dd3-94169bb69f78","Type":"ContainerStarted","Data":"e0a04a5e851c7f7a1e49386e6f45a1056bfd23c2ff4b4b2a3e98b5e57a874798"} Jul 08 18:04:18.936220 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:18.936196 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" event={"ID":"836915b6922e9a09409b13134d65a463","Type":"ContainerStarted","Data":"327baf13a095097e2a8a29c3ac4e598868f32629c78ddfbb847f41b118e28524"} Jul 08 18:04:18.976488 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:18.976431 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-68sht" podStartSLOduration=3.186591416 podStartE2EDuration="13.97640973s" totalImagesPullingTime="10.789818314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:18.953489179 +0000 UTC m=+14.638695947" watchObservedRunningTime="2026-07-08 18:04:18.97640973 +0000 UTC m=+14.661616500" Jul 08 18:04:19.863149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.863051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:19.863149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.863051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:19.863359 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:19.863178 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:19.863359 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:19.863218 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:19.939671 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.939634 2580 generic.go:356] "Generic (PLEG): container finished" podID="be1cf8d52b21daf7b1cf1506ba0db3cf" containerID="5d61a6e37953ca814e7002898efc8e86454a113670948ef0a4e1ee39429d7b3d" exitCode=0 Jul 08 18:04:19.940127 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.939732 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" event={"ID":"be1cf8d52b21daf7b1cf1506ba0db3cf","Type":"ContainerDied","Data":"5d61a6e37953ca814e7002898efc8e86454a113670948ef0a4e1ee39429d7b3d"} Jul 08 18:04:19.941196 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.941165 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t89mc" event={"ID":"7afcc208-15f0-4fc3-9c71-571edc687f52","Type":"ContainerStarted","Data":"c33aec9bbbd6f929d514e1fb4b999511d0040a1616c184b92c97ea6324294fa3"} Jul 08 18:04:19.942744 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.942720 2580 generic.go:356] "Generic (PLEG): container finished" podID="319f6f85-dcd0-4060-89f5-8c32c347ba0d" containerID="edb3bfdfcfa707f8276efa4cb5b35c61e41c8b39b044e5f192c6f11a025fd6a7" exitCode=0 Jul 08 18:04:19.942946 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.942915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerDied","Data":"edb3bfdfcfa707f8276efa4cb5b35c61e41c8b39b044e5f192c6f11a025fd6a7"} Jul 08 18:04:19.944457 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.944431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lngwj" event={"ID":"6d5114bf-8ccf-4e07-8295-8e0d447ca9e5","Type":"ContainerStarted","Data":"de642956516989e48048640face8753ae7a5d8d5456632151709a956553f3802"} Jul 08 18:04:19.946168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.946147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9hvmv" event={"ID":"82f9ee10-112d-4d87-91b3-9619e72c1c03","Type":"ContainerStarted","Data":"2b0f35b91b81a55dddaa6df6b0f2ce378b29247323d5ad4d6ad0446912137328"} Jul 08 18:04:19.947559 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.947533 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tw7m" event={"ID":"5e04054b-06b5-4b90-9891-3e66ab6942c0","Type":"ContainerStarted","Data":"c0b8919d01281d91efed0a81c5943e29506f48c3d08c252679fe26c433ad1340"} Jul 08 18:04:19.949200 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.949178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" event={"ID":"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2","Type":"ContainerStarted","Data":"ed74e782d8b3771c69f40fe2f3230776632eea089f9f95f00417378a3c6fedf3"} Jul 08 18:04:19.956408 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.956366 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-56.ec2.internal" podStartSLOduration=12.956353099 podStartE2EDuration="12.956353099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:18.975926156 +0000 UTC m=+14.661132924" watchObservedRunningTime="2026-07-08 18:04:19.956353099 +0000 UTC m=+15.641559867" Jul 08 18:04:19.971347 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.971308 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9hvmv" podStartSLOduration=4.244449041 podStartE2EDuration="14.971295953s" totalImagesPullingTime="10.726846912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:19.970412982 +0000 UTC m=+15.655619751" watchObservedRunningTime="2026-07-08 18:04:19.971295953 +0000 UTC m=+15.656502722" Jul 08 18:04:19.987133 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:19.987041 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lngwj" podStartSLOduration=4.276421362 podStartE2EDuration="14.987021647s" totalImagesPullingTime="10.710600285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:19.986613094 +0000 UTC m=+15.671819868" watchObservedRunningTime="2026-07-08 18:04:19.987021647 +0000 UTC m=+15.672228417" Jul 08 18:04:20.024660 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:20.024591 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t89mc" podStartSLOduration=3.280631399 podStartE2EDuration="14.024571838s" totalImagesPullingTime="10.743940439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:20.024551246 +0000 UTC m=+15.709758015" watchObservedRunningTime="2026-07-08 18:04:20.024571838 +0000 UTC m=+15.709778605" Jul 08 18:04:20.039340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:20.039301 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5tw7m" podStartSLOduration=3.300013196 podStartE2EDuration="14.039285535s" totalImagesPullingTime="10.739272339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:20.038948485 +0000 UTC m=+15.724155261" watchObservedRunningTime="2026-07-08 18:04:20.039285535 +0000 UTC m=+15.724492304" Jul 08 18:04:20.864134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:20.863860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:20.864333 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:20.864306 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:20.953262 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:20.953217 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" event={"ID":"be1cf8d52b21daf7b1cf1506ba0db3cf","Type":"ContainerStarted","Data":"4611a44734b1f3d7eca3e695bb773f505e03627f9620e121b38497b429e33fc6"} Jul 08 18:04:20.971549 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:20.971496 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-56.ec2.internal" podStartSLOduration=13.971477511 podStartE2EDuration="13.971477511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:20.971002999 +0000 UTC m=+16.656209769" watchObservedRunningTime="2026-07-08 18:04:20.971477511 +0000 UTC m=+16.656684281" Jul 08 18:04:21.212035 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:21.212010 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 18:04:21.795820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:21.795725 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T18:04:21.212030204Z","UUID":"239bca59-8f46-452a-aaa0-024e6925d9bc","Handler":null,"Name":"","Endpoint":""} Jul 08 18:04:21.797182 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:21.797154 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 18:04:21.797306 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:21.797193 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 18:04:21.863052 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:21.863017 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:21.863229 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:21.863029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:21.863229 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:21.863150 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:21.863229 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:21.863213 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:21.957601 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:21.957561 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" event={"ID":"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2","Type":"ContainerStarted","Data":"7311b7248082d3228e99fc816d55564391f277e46e5e81171e2da04822a89f5d"} Jul 08 18:04:22.568289 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:22.568102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:22.568289 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:22.568151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:22.568289 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.568265 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:22.568617 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.568270 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:22.568617 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.568343 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret podName:7ee9becb-c169-4398-b46e-7daa536542a4 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:38.568320717 +0000 UTC m=+34.253527485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret") pod "global-pull-secret-syncer-znlq7" (UID: "7ee9becb-c169-4398-b46e-7daa536542a4") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:22.568617 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.568370 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs podName:21ad1bba-365f-43c5-b06f-13af24af75c3 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:38.568359459 +0000 UTC m=+34.253566210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs") pod "network-metrics-daemon-xvmqm" (UID: "21ad1bba-365f-43c5-b06f-13af24af75c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:22.669024 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:22.668906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:22.669209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.669157 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:22.669209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.669187 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:22.669209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.669201 2580 projected.go:196] Error preparing data for projected volume kube-api-access-79lgv for pod openshift-network-diagnostics/network-check-target-gkkkh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:22.669362 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.669287 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv podName:19db2ce5-00a8-43fa-9927-c747b6fb9d9e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:38.669267033 +0000 UTC m=+34.354473784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-79lgv" (UniqueName: "kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv") pod "network-check-target-gkkkh" (UID: "19db2ce5-00a8-43fa-9927-c747b6fb9d9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:22.863241 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:22.863164 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:22.863387 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:22.863299 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:23.863181 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:23.863138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:23.863181 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:23.863162 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:23.863700 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:23.863289 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:23.863700 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:23.863418 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:24.864190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:24.864155 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:24.864642 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:24.864276 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:24.945377 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:24.945345 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:24.946166 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:24.946147 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:25.863939 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:25.863878 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:25.864127 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:25.864003 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:25.864246 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:25.863881 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:25.864594 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:25.864338 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:26.863557 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:26.863518 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:26.863737 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:26.863661 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:27.864069 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:27.864026 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:27.864595 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:27.864035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:27.864595 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:27.864152 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:27.864595 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:27.864272 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:28.863332 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:28.863297 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:28.863656 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:28.863457 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:29.863540 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:29.863508 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:29.864024 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:29.863550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:29.864024 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:29.863625 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:29.864024 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:29.863742 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:30.863445 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:30.863371 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:30.863601 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:30.863507 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:31.863001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:31.862965 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:31.863178 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:31.863104 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:31.863178 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:31.863161 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:31.863302 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:31.863281 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:32.863630 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.863407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:32.863964 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:32.863734 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:32.982911 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.982881 2580 generic.go:356] "Generic (PLEG): container finished" podID="319f6f85-dcd0-4060-89f5-8c32c347ba0d" containerID="e9c4792fc94fb4c11352fb27efb7ab371113c69eddfd3da12f215a6e4f47070b" exitCode=0 Jul 08 18:04:32.983096 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.982963 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerDied","Data":"e9c4792fc94fb4c11352fb27efb7ab371113c69eddfd3da12f215a6e4f47070b"} Jul 08 18:04:32.987291 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.987261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" event={"ID":"a2561f6e-b1e8-47e9-8aca-14d832e8b3e2","Type":"ContainerStarted","Data":"2d81d66e7c2655646b3b49dd6240063c2751e88498f573e75483249889e0854c"} Jul 08 18:04:32.990598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.990576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"817771745f518d771a56ce66b0ce33458bfc6f3ab9d9e4e5f08dc675a8b98b6d"} Jul 08 18:04:32.990691 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.990651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"adbd092ee2fe45431d2bb5fd50ef10a1b288cdc7fdbb245a8f8e074eeb6c48f1"} Jul 08 18:04:32.990691 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.990685 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"85097f8d8f8930bd8e88e218d38528b5d75ca104c011e4958b5df8ff09618708"} Jul 08 18:04:32.990772 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.990724 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"dc41bb98c6c99990fe6afc7c0fecdb35b228597a0a1a5e755ce8d9c2ef0649c0"} Jul 08 18:04:32.990772 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.990751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"25dbe2d5f21659494a405937ea0ca7e0473fdd3508f0ca4843d602ea2169586b"} Jul 08 18:04:32.990884 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.990789 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"ebc2b20ef7c82e6933d4a11d32ff985328eae31ec4ac19ab9eb34a3c9e382e6d"} Jul 08 18:04:32.992061 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:32.992042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tbfx6" event={"ID":"0d7ae6ce-0d7e-4ac8-8c64-a9f5292ed1cd","Type":"ContainerStarted","Data":"1fce9869b58513faa8bc070a85009d59861fb92b1649483426cc9c0b3a2286e7"} Jul 08 18:04:33.030326 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:33.030273 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tbfx6" podStartSLOduration=3.039462334 podStartE2EDuration="28.030258436s" totalImagesPullingTime="24.990796102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:33.029869086 +0000 UTC m=+28.715075855" watchObservedRunningTime="2026-07-08 18:04:33.030258436 +0000 UTC m=+28.715465204" Jul 08 18:04:33.863938 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:33.863909 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:33.864433 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:33.864018 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:33.864433 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:33.864070 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:33.864579 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:33.864432 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:34.105693 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:34.105655 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:34.105918 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:34.105817 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 18:04:34.106264 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:34.106236 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9hvmv" Jul 08 18:04:34.124492 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:34.124407 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jrpqt" podStartSLOduration=5.661026833 podStartE2EDuration="29.124392534s" totalImagesPullingTime="23.463365701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:33.049031692 +0000 UTC m=+28.734238471" watchObservedRunningTime="2026-07-08 18:04:34.124392534 +0000 UTC m=+29.809599304" Jul 08 18:04:34.863930 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:34.863827 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:34.864129 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:34.863933 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:34.998015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:34.997987 2580 generic.go:356] "Generic (PLEG): container finished" podID="319f6f85-dcd0-4060-89f5-8c32c347ba0d" containerID="d9282208e93a616f52cbc075e78232a2b9b44439951febf59d28a8b30d9355da" exitCode=0 Jul 08 18:04:34.998128 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:34.998074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerDied","Data":"d9282208e93a616f52cbc075e78232a2b9b44439951febf59d28a8b30d9355da"} Jul 08 18:04:35.863690 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:35.863646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:35.863690 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:35.863690 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:35.863986 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:35.863761 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:35.863986 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:35.863888 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:36.003449 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:36.003417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"22f76bf86160efb084ae9073d687609ef15d32517f6e611e7e8d3ee59404c0ed"} Jul 08 18:04:36.863427 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:36.863393 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:36.863594 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:36.863502 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:37.008025 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:37.007989 2580 generic.go:356] "Generic (PLEG): container finished" podID="319f6f85-dcd0-4060-89f5-8c32c347ba0d" containerID="aae8056f3a065e11dd61f67f0a9e924d055f2bb0ae62eddca092a0e1ae3603a3" exitCode=0 Jul 08 18:04:37.008408 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:37.008055 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerDied","Data":"aae8056f3a065e11dd61f67f0a9e924d055f2bb0ae62eddca092a0e1ae3603a3"} Jul 08 18:04:37.863592 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:37.863401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:37.863788 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:37.863459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:37.863788 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:37.863679 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:37.863788 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:37.863760 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:38.013523 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.013492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" event={"ID":"e034a2a0-5921-4aeb-8e0a-9ca4f1994be5","Type":"ContainerStarted","Data":"ee8e5c26b12c94d10f6ec831b87bc08df797dac3c89b005c315ffda6bcbb2be5"} Jul 08 18:04:38.014021 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.013718 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:38.028808 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.028784 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:38.047454 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.047409 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" podStartSLOduration=7.035945773 podStartE2EDuration="32.047393395s" totalImagesPullingTime="25.011447622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:38.046868937 +0000 UTC m=+33.732075711" watchObservedRunningTime="2026-07-08 18:04:38.047393395 +0000 UTC m=+33.732600164" Jul 08 18:04:38.592958 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.592905 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:38.593154 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.592968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:38.593154 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.593085 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:38.593154 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.593114 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:38.593309 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.593156 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret podName:7ee9becb-c169-4398-b46e-7daa536542a4 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:10.593139605 +0000 UTC m=+66.278346353 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret") pod "global-pull-secret-syncer-znlq7" (UID: "7ee9becb-c169-4398-b46e-7daa536542a4") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:38.593309 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.593172 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs podName:21ad1bba-365f-43c5-b06f-13af24af75c3 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:10.593165219 +0000 UTC m=+66.278371966 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs") pod "network-metrics-daemon-xvmqm" (UID: "21ad1bba-365f-43c5-b06f-13af24af75c3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:38.694170 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.694129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:38.694314 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.694251 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:38.694314 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.694264 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:38.694314 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.694273 2580 projected.go:196] Error preparing data for projected volume kube-api-access-79lgv for pod openshift-network-diagnostics/network-check-target-gkkkh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:38.694399 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.694320 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv podName:19db2ce5-00a8-43fa-9927-c747b6fb9d9e nodeName:}" failed. No retries permitted until 2026-07-08 18:05:10.69430824 +0000 UTC m=+66.379514987 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-79lgv" (UniqueName: "kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv") pod "network-check-target-gkkkh" (UID: "19db2ce5-00a8-43fa-9927-c747b6fb9d9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:38.863484 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:38.863398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:38.863628 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:38.863525 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:39.016374 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.016340 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:39.016855 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.016411 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:39.030519 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.030492 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:04:39.408567 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.408523 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gkkkh"] Jul 08 18:04:39.408747 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.408685 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:39.408859 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:39.408818 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:39.412621 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.412595 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-znlq7"] Jul 08 18:04:39.412761 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.412689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:39.412815 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:39.412772 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:39.414790 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.414766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xvmqm"] Jul 08 18:04:39.414950 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:39.414901 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:39.415026 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:39.415001 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:40.863889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:40.863854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:40.864334 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:40.863868 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:40.864334 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:40.863987 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:40.864334 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:40.864014 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:40.864334 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:40.864123 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:40.864334 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:40.864208 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:42.863568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:42.863530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:42.863956 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:42.863530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:42.863956 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:42.863663 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:42.863956 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:42.863661 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:42.863956 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:42.863793 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:42.863956 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:42.863909 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:44.864468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:44.864425 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:44.864468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:44.864468 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:44.864950 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:44.864523 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkkkh" podUID="19db2ce5-00a8-43fa-9927-c747b6fb9d9e" Jul 08 18:04:44.864950 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:44.864569 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-znlq7" podUID="7ee9becb-c169-4398-b46e-7daa536542a4" Jul 08 18:04:44.864950 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:44.864615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:44.864950 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:44.864730 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvmqm" podUID="21ad1bba-365f-43c5-b06f-13af24af75c3" Jul 08 18:04:45.545310 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.545163 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-56.ec2.internal" event="NodeReady" Jul 08 18:04:45.545473 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.545405 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 18:04:45.583192 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.583165 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d459b6969-8sg97"] Jul 08 18:04:45.593019 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.592990 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85fd9b879b-6xmsl"] Jul 08 18:04:45.593121 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.593105 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.595611 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.595589 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 18:04:45.595720 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.595701 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.596177 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.596159 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 18:04:45.596287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.596228 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 18:04:45.596381 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.596365 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hgp88\"" Jul 08 18:04:45.596632 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.596616 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 18:04:45.622007 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.621982 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.623423 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.623399 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-lnv6h"] Jul 08 18:04:45.623574 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.623541 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.632258 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.632234 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 18:04:45.636886 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.636864 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9qxwl\"" Jul 08 18:04:45.637009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.636939 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 18:04:45.638247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.638228 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 18:04:45.641465 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.641407 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s"] Jul 08 18:04:45.641572 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.641531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.650263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4f7d7b3b-c770-4330-ba51-218d3b049f6b-snapshots\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.650263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f7d7b3b-c770-4330-ba51-218d3b049f6b-serving-cert\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.650263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-trusted-ca\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-bound-sa-token\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7d7b3b-c770-4330-ba51-218d3b049f6b-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-default-certificate\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650308 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-image-registry-private-configuration\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6457a1b-e229-424b-a634-78951adc295f-ca-trust-extracted\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f7d7b3b-c770-4330-ba51-218d3b049f6b-tmp\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650406 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650470 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vkf\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-kube-api-access-z5vkf\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.650598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-registry-certificates\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2g5m\" (UniqueName: \"kubernetes.io/projected/4f7d7b3b-c770-4330-ba51-218d3b049f6b-kube-api-access-f2g5m\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.650913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-installation-pull-secrets\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.650913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-stats-auth\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.650913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650667 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nvd4\" (UniqueName: \"kubernetes.io/projected/6c74951d-5706-4a46-9163-5b9d3dc67042-kube-api-access-4nvd4\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.650913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.650709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7d7b3b-c770-4330-ba51-218d3b049f6b-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.651677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.651473 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 18:04:45.651677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.651501 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.651677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.651553 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.651677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.651473 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 18:04:45.651947 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.651914 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jcf9j\"" Jul 08 18:04:45.663530 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.663505 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg"] Jul 08 18:04:45.663644 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.663607 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:45.667476 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.667389 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 18:04:45.677000 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.676846 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 08 18:04:45.677000 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.676866 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.677169 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.677113 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.683663 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.682888 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4bt25\"" Jul 08 18:04:45.685366 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.685286 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 18:04:45.685708 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.685690 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf"] Jul 08 18:04:45.685788 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.685775 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.693644 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.693624 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.694072 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.694048 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.694322 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.694301 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 18:04:45.694322 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.694311 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lt6tk\"" Jul 08 18:04:45.697201 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.697181 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 18:04:45.705911 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.705880 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x"] Jul 08 18:04:45.706012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.705997 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" Jul 08 18:04:45.709996 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.709947 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.710100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.710006 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.710100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.710038 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kcng7\"" Jul 08 18:04:45.720059 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.720040 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4vx47"] Jul 08 18:04:45.720184 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.720168 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.722739 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.722724 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 18:04:45.723232 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.723217 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.723312 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.723297 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.723524 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.723510 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-g8ntv\"" Jul 08 18:04:45.723598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.723584 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 18:04:45.730045 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.730030 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw"] Jul 08 18:04:45.730149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.730135 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.732346 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.732330 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.732427 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.732331 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 18:04:45.732719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.732707 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.732867 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.732828 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 18:04:45.733674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.733660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-42tnt\"" Jul 08 18:04:45.737501 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.737481 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 18:04:45.742182 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.742167 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-jq98v"] Jul 08 18:04:45.742309 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.742295 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.746237 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.746134 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.746524 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.746504 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mjh89\"" Jul 08 18:04:45.746613 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.746540 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.746613 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.746509 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 18:04:45.747608 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.747593 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 18:04:45.751058 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-installation-pull-secrets\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.751124 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:45.751124 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hrhb\" (UniqueName: \"kubernetes.io/projected/f0f85e84-3171-495f-a6f2-3d9def5ccc1c-kube-api-access-4hrhb\") pod \"volume-data-source-validator-5d456fdbcd-pnrdf\" (UID: \"f0f85e84-3171-495f-a6f2-3d9def5ccc1c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" Jul 08 18:04:45.751188 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c57f641-7b79-42b9-a796-35eb35a22fc6-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.751240 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7d7b3b-c770-4330-ba51-218d3b049f6b-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.751287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751270 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.751372 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4f7d7b3b-c770-4330-ba51-218d3b049f6b-snapshots\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.751412 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.751376 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:46.251356224 +0000 UTC m=+41.936563010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : configmap references non-existent config key: service-ca.crt Jul 08 18:04:45.751468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-bound-sa-token\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.751468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7d7b3b-c770-4330-ba51-218d3b049f6b-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.751560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751466 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-default-certificate\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.751560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6457a1b-e229-424b-a634-78951adc295f-ca-trust-extracted\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.751560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f7d7b3b-c770-4330-ba51-218d3b049f6b-tmp\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.751560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751549 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6525a595-633b-454a-931a-5c2c55f5b8b4-serving-cert\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.751749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vkf\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-kube-api-access-z5vkf\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.751749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26vxl\" (UniqueName: \"kubernetes.io/projected/7c57f641-7b79-42b9-a796-35eb35a22fc6-kube-api-access-26vxl\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.751749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3fddd715-cce6-4e4d-a611-adc89869fd3e-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.751749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751684 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-registry-certificates\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.751749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5195670-b3c0-4736-a5dc-3faab67e0195-config\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.751749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2zj8\" (UniqueName: \"kubernetes.io/projected/a5195670-b3c0-4736-a5dc-3faab67e0195-kube-api-access-v2zj8\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2g5m\" (UniqueName: \"kubernetes.io/projected/4f7d7b3b-c770-4330-ba51-218d3b049f6b-kube-api-access-f2g5m\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-stats-auth\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751827 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7d7b3b-c770-4330-ba51-218d3b049f6b-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nvd4\" (UniqueName: \"kubernetes.io/projected/6c74951d-5706-4a46-9163-5b9d3dc67042-kube-api-access-4nvd4\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5195670-b3c0-4736-a5dc-3faab67e0195-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751935 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c57f641-7b79-42b9-a796-35eb35a22fc6-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751945 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4f7d7b3b-c770-4330-ba51-218d3b049f6b-snapshots\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.751963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.752029 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:04:45.752074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6457a1b-e229-424b-a634-78951adc295f-ca-trust-extracted\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.752085 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:46.252070086 +0000 UTC m=+41.937276846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : secret "router-metrics-certs-default" not found Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f7d7b3b-c770-4330-ba51-218d3b049f6b-serving-cert\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752313 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f7d7b3b-c770-4330-ba51-218d3b049f6b-tmp\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752328 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6525a595-633b-454a-931a-5c2c55f5b8b4-trusted-ca\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lsc8\" (UniqueName: \"kubernetes.io/projected/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-kube-api-access-2lsc8\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.752458 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.752471 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd9b879b-6xmsl: secret "image-registry-tls" not found Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752482 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7d7b3b-c770-4330-ba51-218d3b049f6b-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.752510 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls podName:b6457a1b-e229-424b-a634-78951adc295f nodeName:}" failed. No retries permitted until 2026-07-08 18:04:46.252495285 +0000 UTC m=+41.937702042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls") pod "image-registry-85fd9b879b-6xmsl" (UID: "b6457a1b-e229-424b-a634-78951adc295f") : secret "image-registry-tls" not found Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsmb7\" (UniqueName: \"kubernetes.io/projected/3fddd715-cce6-4e4d-a611-adc89869fd3e-kube-api-access-dsmb7\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.752603 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-trusted-ca\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.753179 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752636 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-image-registry-private-configuration\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.753179 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6525a595-633b-454a-931a-5c2c55f5b8b4-config\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.753179 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752716 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g88rk\" (UniqueName: \"kubernetes.io/projected/6525a595-633b-454a-931a-5c2c55f5b8b4-kube-api-access-g88rk\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.753179 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.752850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-registry-certificates\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.753419 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.753393 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-trusted-ca\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.755134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.754237 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d459b6969-8sg97"] Jul 08 18:04:45.755134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.754272 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-57w86"] Jul 08 18:04:45.755134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.754348 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:45.756039 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.756014 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f7d7b3b-c770-4330-ba51-218d3b049f6b-serving-cert\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.756134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.756041 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-image-registry-private-configuration\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.756134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.756064 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-installation-pull-secrets\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.756134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.756075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-default-certificate\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.756342 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.756322 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-stats-auth\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.760153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.757179 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 18:04:45.760153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.757282 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jfbrc\"" Jul 08 18:04:45.760153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.757439 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 18:04:45.762496 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.762476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nvd4\" (UniqueName: \"kubernetes.io/projected/6c74951d-5706-4a46-9163-5b9d3dc67042-kube-api-access-4nvd4\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:45.762673 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.762653 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vkf\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-kube-api-access-z5vkf\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.763485 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.763470 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-bound-sa-token\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:45.763717 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.763698 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2g5m\" (UniqueName: \"kubernetes.io/projected/4f7d7b3b-c770-4330-ba51-218d3b049f6b-kube-api-access-f2g5m\") pod \"insights-operator-5c847dcfc4-lnv6h\" (UID: \"4f7d7b3b-c770-4330-ba51-218d3b049f6b\") " pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.777122 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.777099 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s"] Jul 08 18:04:45.777217 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.777141 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xsrcw"] Jul 08 18:04:45.777217 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.777203 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" Jul 08 18:04:45.779726 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.779700 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.780031 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.779702 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rd8vr\"" Jul 08 18:04:45.780031 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.779739 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.793082 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.793060 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tj2cs"] Jul 08 18:04:45.793177 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.793164 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:45.795725 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.795705 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 18:04:45.796072 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.796051 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 18:04:45.796161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.796092 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 18:04:45.796224 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.796178 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cd4hg\"" Jul 08 18:04:45.804976 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.804955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg"] Jul 08 18:04:45.805060 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.804989 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x"] Jul 08 18:04:45.805060 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805012 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4vx47"] Jul 08 18:04:45.805060 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805032 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw"] Jul 08 18:04:45.805060 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805048 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85fd9b879b-6xmsl"] Jul 08 18:04:45.805192 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805064 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.805192 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805065 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-57w86"] Jul 08 18:04:45.805282 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805202 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf"] Jul 08 18:04:45.805282 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805226 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-jq98v"] Jul 08 18:04:45.805282 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805246 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tj2cs"] Jul 08 18:04:45.805282 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805269 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xsrcw"] Jul 08 18:04:45.805399 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.805294 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-lnv6h"] Jul 08 18:04:45.807752 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.807739 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmswz\"" Jul 08 18:04:45.807752 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.807746 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 18:04:45.807883 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.807752 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 18:04:45.853987 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.853962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6525a595-633b-454a-931a-5c2c55f5b8b4-serving-cert\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.854153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.854153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854025 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26vxl\" (UniqueName: \"kubernetes.io/projected/7c57f641-7b79-42b9-a796-35eb35a22fc6-kube-api-access-26vxl\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.854153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3fddd715-cce6-4e4d-a611-adc89869fd3e-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.854153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854060 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:45.854153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5195670-b3c0-4736-a5dc-3faab67e0195-config\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.854418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854184 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2zj8\" (UniqueName: \"kubernetes.io/projected/a5195670-b3c0-4736-a5dc-3faab67e0195-kube-api-access-v2zj8\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.854418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/084a77c4-cf51-49c0-8cb0-1963bec63658-config-volume\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.854418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xzxg\" (UniqueName: \"kubernetes.io/projected/80a3f6d5-6f6d-42ef-861a-76a39a41a027-kube-api-access-8xzxg\") pod \"network-check-source-989f8b8dc-57w86\" (UID: \"80a3f6d5-6f6d-42ef-861a-76a39a41a027\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" Jul 08 18:04:45.854418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5195670-b3c0-4736-a5dc-3faab67e0195-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.854418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c57f641-7b79-42b9-a796-35eb35a22fc6-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.854418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6525a595-633b-454a-931a-5c2c55f5b8b4-trusted-ca\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lsc8\" (UniqueName: \"kubernetes.io/projected/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-kube-api-access-2lsc8\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsmb7\" (UniqueName: \"kubernetes.io/projected/3fddd715-cce6-4e4d-a611-adc89869fd3e-kube-api-access-dsmb7\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084a77c4-cf51-49c0-8cb0-1963bec63658-tmp-dir\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4mdw\" (UniqueName: \"kubernetes.io/projected/084a77c4-cf51-49c0-8cb0-1963bec63658-kube-api-access-x4mdw\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854549 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76d17d44-d39f-4b04-9443-28a71704f3a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c398807-d5ee-4ab3-9cf6-94364606b1d6-cert\") pod \"ingress-canary-xsrcw\" (UID: \"3c398807-d5ee-4ab3-9cf6-94364606b1d6\") " pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854606 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5195670-b3c0-4736-a5dc-3faab67e0195-config\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6525a595-633b-454a-931a-5c2c55f5b8b4-config\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g88rk\" (UniqueName: \"kubernetes.io/projected/6525a595-633b-454a-931a-5c2c55f5b8b4-kube-api-access-g88rk\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.854710 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:45.854729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hrhb\" (UniqueName: \"kubernetes.io/projected/f0f85e84-3171-495f-a6f2-3d9def5ccc1c-kube-api-access-4hrhb\") pod \"volume-data-source-validator-5d456fdbcd-pnrdf\" (UID: \"f0f85e84-3171-495f-a6f2-3d9def5ccc1c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" Jul 08 18:04:45.855407 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c57f641-7b79-42b9-a796-35eb35a22fc6-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.855407 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.854780 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls podName:3fddd715-cce6-4e4d-a611-adc89869fd3e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:46.354760861 +0000 UTC m=+42.039967614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-clrnw" (UID: "3fddd715-cce6-4e4d-a611-adc89869fd3e") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:45.855407 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854798 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3fddd715-cce6-4e4d-a611-adc89869fd3e-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.855407 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.854810 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvh2j\" (UniqueName: \"kubernetes.io/projected/3c398807-d5ee-4ab3-9cf6-94364606b1d6-kube-api-access-xvh2j\") pod \"ingress-canary-xsrcw\" (UID: \"3c398807-d5ee-4ab3-9cf6-94364606b1d6\") " pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:45.855407 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.855009 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 18:04:45.855407 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.855079 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls podName:7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c nodeName:}" failed. No retries permitted until 2026-07-08 18:04:46.355054172 +0000 UTC m=+42.040260935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-jsp8s" (UID: "7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c") : secret "samples-operator-tls" not found Jul 08 18:04:45.855407 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.855311 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c57f641-7b79-42b9-a796-35eb35a22fc6-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.855616 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.855432 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6525a595-633b-454a-931a-5c2c55f5b8b4-trusted-ca\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.855616 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.855466 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6525a595-633b-454a-931a-5c2c55f5b8b4-config\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.857216 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.857195 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c57f641-7b79-42b9-a796-35eb35a22fc6-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.857285 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.857220 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5195670-b3c0-4736-a5dc-3faab67e0195-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.857414 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.857395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6525a595-633b-454a-931a-5c2c55f5b8b4-serving-cert\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.867584 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.867555 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsmb7\" (UniqueName: \"kubernetes.io/projected/3fddd715-cce6-4e4d-a611-adc89869fd3e-kube-api-access-dsmb7\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:45.870721 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.870701 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26vxl\" (UniqueName: \"kubernetes.io/projected/7c57f641-7b79-42b9-a796-35eb35a22fc6-kube-api-access-26vxl\") pod \"kube-storage-version-migrator-operator-846bf668f7-wf66x\" (UID: \"7c57f641-7b79-42b9-a796-35eb35a22fc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:45.870960 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.870936 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hrhb\" (UniqueName: \"kubernetes.io/projected/f0f85e84-3171-495f-a6f2-3d9def5ccc1c-kube-api-access-4hrhb\") pod \"volume-data-source-validator-5d456fdbcd-pnrdf\" (UID: \"f0f85e84-3171-495f-a6f2-3d9def5ccc1c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" Jul 08 18:04:45.871596 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.871574 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2zj8\" (UniqueName: \"kubernetes.io/projected/a5195670-b3c0-4736-a5dc-3faab67e0195-kube-api-access-v2zj8\") pod \"service-ca-operator-5d46d9ffd4-6rsgg\" (UID: \"a5195670-b3c0-4736-a5dc-3faab67e0195\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:45.872456 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.872432 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lsc8\" (UniqueName: \"kubernetes.io/projected/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-kube-api-access-2lsc8\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:45.872530 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.872461 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g88rk\" (UniqueName: \"kubernetes.io/projected/6525a595-633b-454a-931a-5c2c55f5b8b4-kube-api-access-g88rk\") pod \"console-operator-68b757fb67-4vx47\" (UID: \"6525a595-633b-454a-931a-5c2c55f5b8b4\") " pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:45.953277 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.953240 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" Jul 08 18:04:45.955553 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084a77c4-cf51-49c0-8cb0-1963bec63658-tmp-dir\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.955669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955559 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4mdw\" (UniqueName: \"kubernetes.io/projected/084a77c4-cf51-49c0-8cb0-1963bec63658-kube-api-access-x4mdw\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.955669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76d17d44-d39f-4b04-9443-28a71704f3a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:45.955669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c398807-d5ee-4ab3-9cf6-94364606b1d6-cert\") pod \"ingress-canary-xsrcw\" (UID: \"3c398807-d5ee-4ab3-9cf6-94364606b1d6\") " pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:45.955820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955683 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvh2j\" (UniqueName: \"kubernetes.io/projected/3c398807-d5ee-4ab3-9cf6-94364606b1d6-kube-api-access-xvh2j\") pod \"ingress-canary-xsrcw\" (UID: \"3c398807-d5ee-4ab3-9cf6-94364606b1d6\") " pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:45.955820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.955820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:45.956012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/084a77c4-cf51-49c0-8cb0-1963bec63658-config-volume\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.956012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xzxg\" (UniqueName: \"kubernetes.io/projected/80a3f6d5-6f6d-42ef-861a-76a39a41a027-kube-api-access-8xzxg\") pod \"network-check-source-989f8b8dc-57w86\" (UID: \"80a3f6d5-6f6d-42ef-861a-76a39a41a027\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" Jul 08 18:04:45.956012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.955944 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/084a77c4-cf51-49c0-8cb0-1963bec63658-tmp-dir\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.956012 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.955949 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:04:45.956012 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.955993 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 18:04:45.956251 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.956046 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert podName:76d17d44-d39f-4b04-9443-28a71704f3a8 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:46.456028985 +0000 UTC m=+42.141235757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-jq98v" (UID: "76d17d44-d39f-4b04-9443-28a71704f3a8") : secret "networking-console-plugin-cert" not found Jul 08 18:04:45.956251 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:45.956065 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls podName:084a77c4-cf51-49c0-8cb0-1963bec63658 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:46.456054556 +0000 UTC m=+42.141261307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls") pod "dns-default-tj2cs" (UID: "084a77c4-cf51-49c0-8cb0-1963bec63658") : secret "dns-default-metrics-tls" not found Jul 08 18:04:45.957646 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.956591 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76d17d44-d39f-4b04-9443-28a71704f3a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:45.957646 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.956800 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/084a77c4-cf51-49c0-8cb0-1963bec63658-config-volume\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.958786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.958767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c398807-d5ee-4ab3-9cf6-94364606b1d6-cert\") pod \"ingress-canary-xsrcw\" (UID: \"3c398807-d5ee-4ab3-9cf6-94364606b1d6\") " pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:45.964906 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.964856 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4mdw\" (UniqueName: \"kubernetes.io/projected/084a77c4-cf51-49c0-8cb0-1963bec63658-kube-api-access-x4mdw\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:45.965630 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.965610 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvh2j\" (UniqueName: \"kubernetes.io/projected/3c398807-d5ee-4ab3-9cf6-94364606b1d6-kube-api-access-xvh2j\") pod \"ingress-canary-xsrcw\" (UID: \"3c398807-d5ee-4ab3-9cf6-94364606b1d6\") " pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:45.965701 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.965614 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xzxg\" (UniqueName: \"kubernetes.io/projected/80a3f6d5-6f6d-42ef-861a-76a39a41a027-kube-api-access-8xzxg\") pod \"network-check-source-989f8b8dc-57w86\" (UID: \"80a3f6d5-6f6d-42ef-861a-76a39a41a027\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" Jul 08 18:04:45.997539 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:45.997504 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" Jul 08 18:04:46.032910 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.032882 2580 generic.go:356] "Generic (PLEG): container finished" podID="319f6f85-dcd0-4060-89f5-8c32c347ba0d" containerID="d9753a42f77648390af3c858a999113ed8eff2352973cca5a1ba48a9e228906b" exitCode=0 Jul 08 18:04:46.033038 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.032918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerDied","Data":"d9753a42f77648390af3c858a999113ed8eff2352973cca5a1ba48a9e228906b"} Jul 08 18:04:46.078015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.077573 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" Jul 08 18:04:46.083774 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.083719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" Jul 08 18:04:46.089031 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.088944 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:46.107001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.106966 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" Jul 08 18:04:46.115953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.113238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xsrcw" Jul 08 18:04:46.155563 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.155485 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-lnv6h"] Jul 08 18:04:46.158621 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:46.158584 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7d7b3b_c770_4330_ba51_218d3b049f6b.slice/crio-0f2a32976e741b91b04913b5a77ad28978e4e0ff408a1833b57f80cadb9fe135 WatchSource:0}: Error finding container 0f2a32976e741b91b04913b5a77ad28978e4e0ff408a1833b57f80cadb9fe135: Status 404 returned error can't find the container with id 0f2a32976e741b91b04913b5a77ad28978e4e0ff408a1833b57f80cadb9fe135 Jul 08 18:04:46.171991 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.171731 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg"] Jul 08 18:04:46.181015 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:46.180971 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5195670_b3c0_4736_a5dc_3faab67e0195.slice/crio-f206b67df7b29ffe8e0ad79930bae7ef09fee866fb63381491f6162904e7ece0 WatchSource:0}: Error finding container f206b67df7b29ffe8e0ad79930bae7ef09fee866fb63381491f6162904e7ece0: Status 404 returned error can't find the container with id f206b67df7b29ffe8e0ad79930bae7ef09fee866fb63381491f6162904e7ece0 Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.261638 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.261769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.261799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.262021 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.262040 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd9b879b-6xmsl: secret "image-registry-tls" not found Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.262097 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls podName:b6457a1b-e229-424b-a634-78951adc295f nodeName:}" failed. No retries permitted until 2026-07-08 18:04:47.262079046 +0000 UTC m=+42.947285799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls") pod "image-registry-85fd9b879b-6xmsl" (UID: "b6457a1b-e229-424b-a634-78951adc295f") : secret "image-registry-tls" not found Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.262538 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:47.262521829 +0000 UTC m=+42.947728578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : configmap references non-existent config key: service-ca.crt Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.262606 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:04:46.262682 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.262640 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:47.262628698 +0000 UTC m=+42.947835445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : secret "router-metrics-certs-default" not found Jul 08 18:04:46.289377 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.289340 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf"] Jul 08 18:04:46.315117 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.315026 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xsrcw"] Jul 08 18:04:46.331750 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.331649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x"] Jul 08 18:04:46.362560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.362529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:46.362693 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.362589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:46.362693 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.362680 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:46.362757 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.362734 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 18:04:46.362757 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.362746 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls podName:3fddd715-cce6-4e4d-a611-adc89869fd3e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:47.362730999 +0000 UTC m=+43.047937746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-clrnw" (UID: "3fddd715-cce6-4e4d-a611-adc89869fd3e") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:46.362857 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.362783 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls podName:7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c nodeName:}" failed. No retries permitted until 2026-07-08 18:04:47.362769562 +0000 UTC m=+43.047976309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-jsp8s" (UID: "7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c") : secret "samples-operator-tls" not found Jul 08 18:04:46.386599 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.386474 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-57w86"] Jul 08 18:04:46.387444 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.387423 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4vx47"] Jul 08 18:04:46.389116 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:46.389086 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a3f6d5_6f6d_42ef_861a_76a39a41a027.slice/crio-d09c782fce7c3c5df4ed54987a2c467da6a8d4e15d6ea720ab681f61788f1976 WatchSource:0}: Error finding container d09c782fce7c3c5df4ed54987a2c467da6a8d4e15d6ea720ab681f61788f1976: Status 404 returned error can't find the container with id d09c782fce7c3c5df4ed54987a2c467da6a8d4e15d6ea720ab681f61788f1976 Jul 08 18:04:46.389807 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:46.389752 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c398807_d5ee_4ab3_9cf6_94364606b1d6.slice/crio-c200717123066b3f8456d1d60717b1194e3673edfb27190c76ac501ff10befaa WatchSource:0}: Error finding container c200717123066b3f8456d1d60717b1194e3673edfb27190c76ac501ff10befaa: Status 404 returned error can't find the container with id c200717123066b3f8456d1d60717b1194e3673edfb27190c76ac501ff10befaa Jul 08 18:04:46.390791 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:46.390724 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f85e84_3171_495f_a6f2_3d9def5ccc1c.slice/crio-2857df357f0669f03bee9a2009b1e7aa473ba7c45c0f7ceb6c6132d2f6faf414 WatchSource:0}: Error finding container 2857df357f0669f03bee9a2009b1e7aa473ba7c45c0f7ceb6c6132d2f6faf414: Status 404 returned error can't find the container with id 2857df357f0669f03bee9a2009b1e7aa473ba7c45c0f7ceb6c6132d2f6faf414 Jul 08 18:04:46.391968 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:46.391880 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6525a595_633b_454a_931a_5c2c55f5b8b4.slice/crio-18c4e15f1c5e6adf79508842f1a0b63277d509a66ff1293f76ff1b8c39cbe16d WatchSource:0}: Error finding container 18c4e15f1c5e6adf79508842f1a0b63277d509a66ff1293f76ff1b8c39cbe16d: Status 404 returned error can't find the container with id 18c4e15f1c5e6adf79508842f1a0b63277d509a66ff1293f76ff1b8c39cbe16d Jul 08 18:04:46.392485 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:46.392452 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c57f641_7b79_42b9_a796_35eb35a22fc6.slice/crio-3892f650697e20e2d5763d8238e79cb8cb84a7a3cea607a1372120e7f8b0de8d WatchSource:0}: Error finding container 3892f650697e20e2d5763d8238e79cb8cb84a7a3cea607a1372120e7f8b0de8d: Status 404 returned error can't find the container with id 3892f650697e20e2d5763d8238e79cb8cb84a7a3cea607a1372120e7f8b0de8d Jul 08 18:04:46.463365 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.463328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:46.463541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.463380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:46.463541 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.463431 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:04:46.463541 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.463504 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls podName:084a77c4-cf51-49c0-8cb0-1963bec63658 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:47.463488025 +0000 UTC m=+43.148694775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls") pod "dns-default-tj2cs" (UID: "084a77c4-cf51-49c0-8cb0-1963bec63658") : secret "dns-default-metrics-tls" not found Jul 08 18:04:46.463541 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.463527 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 18:04:46.463709 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:46.463591 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert podName:76d17d44-d39f-4b04-9443-28a71704f3a8 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:47.463575068 +0000 UTC m=+43.148781826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-jq98v" (UID: "76d17d44-d39f-4b04-9443-28a71704f3a8") : secret "networking-console-plugin-cert" not found Jul 08 18:04:46.484866 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.484761 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:46.584517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.584479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:46.648919 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.648867 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:46.705427 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.705393 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:46.807543 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.807466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:46.867054 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.867022 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:04:46.867195 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.867135 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:04:46.867195 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.867161 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:04:46.870000 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.869982 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d9s8r\"" Jul 08 18:04:46.870391 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.870062 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8pfq\"" Jul 08 18:04:46.870503 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.870488 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:04:46.870654 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:46.870640 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:04:47.037735 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.037600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" event={"ID":"80a3f6d5-6f6d-42ef-861a-76a39a41a027","Type":"ContainerStarted","Data":"d09c782fce7c3c5df4ed54987a2c467da6a8d4e15d6ea720ab681f61788f1976"} Jul 08 18:04:47.046792 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.044298 2580 generic.go:356] "Generic (PLEG): container finished" podID="319f6f85-dcd0-4060-89f5-8c32c347ba0d" containerID="cdda938ba82fe08f4daa950bd0e95f0ec56a6f30cfb5bb224fd5cb18597a89ac" exitCode=0 Jul 08 18:04:47.046792 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.044423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerDied","Data":"cdda938ba82fe08f4daa950bd0e95f0ec56a6f30cfb5bb224fd5cb18597a89ac"} Jul 08 18:04:47.051364 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.051341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" event={"ID":"a5195670-b3c0-4736-a5dc-3faab67e0195","Type":"ContainerStarted","Data":"f206b67df7b29ffe8e0ad79930bae7ef09fee866fb63381491f6162904e7ece0"} Jul 08 18:04:47.053192 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.053167 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xsrcw" event={"ID":"3c398807-d5ee-4ab3-9cf6-94364606b1d6","Type":"ContainerStarted","Data":"c200717123066b3f8456d1d60717b1194e3673edfb27190c76ac501ff10befaa"} Jul 08 18:04:47.054867 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.054827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" event={"ID":"7c57f641-7b79-42b9-a796-35eb35a22fc6","Type":"ContainerStarted","Data":"3892f650697e20e2d5763d8238e79cb8cb84a7a3cea607a1372120e7f8b0de8d"} Jul 08 18:04:47.056085 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.056065 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" event={"ID":"f0f85e84-3171-495f-a6f2-3d9def5ccc1c","Type":"ContainerStarted","Data":"2857df357f0669f03bee9a2009b1e7aa473ba7c45c0f7ceb6c6132d2f6faf414"} Jul 08 18:04:47.057526 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.057500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" event={"ID":"6525a595-633b-454a-931a-5c2c55f5b8b4","Type":"ContainerStarted","Data":"18c4e15f1c5e6adf79508842f1a0b63277d509a66ff1293f76ff1b8c39cbe16d"} Jul 08 18:04:47.058807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.058741 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" event={"ID":"4f7d7b3b-c770-4330-ba51-218d3b049f6b","Type":"ContainerStarted","Data":"0f2a32976e741b91b04913b5a77ad28978e4e0ff408a1833b57f80cadb9fe135"} Jul 08 18:04:47.221934 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.221631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.270187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.270312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.270343 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.270501 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.270516 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd9b879b-6xmsl: secret "image-registry-tls" not found Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.270570 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.270553408 +0000 UTC m=+44.955760168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : configmap references non-existent config key: service-ca.crt Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.270592 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls podName:b6457a1b-e229-424b-a634-78951adc295f nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.270584003 +0000 UTC m=+44.955790750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls") pod "image-registry-85fd9b879b-6xmsl" (UID: "b6457a1b-e229-424b-a634-78951adc295f") : secret "image-registry-tls" not found Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.270655 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:04:47.270777 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.270689 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.270679558 +0000 UTC m=+44.955886328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : secret "router-metrics-certs-default" not found Jul 08 18:04:47.320769 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.320620 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:47.371366 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.371327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:47.371571 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.371548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:47.372384 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.371699 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:47.372384 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.371768 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls podName:3fddd715-cce6-4e4d-a611-adc89869fd3e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.371750241 +0000 UTC m=+45.056956992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-clrnw" (UID: "3fddd715-cce6-4e4d-a611-adc89869fd3e") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:47.372384 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.372246 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 18:04:47.372384 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.372299 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls podName:7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.372281647 +0000 UTC m=+45.057488397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-jsp8s" (UID: "7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c") : secret "samples-operator-tls" not found Jul 08 18:04:47.473209 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.472401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:47.473209 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.472444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:47.473209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.472643 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 18:04:47.473209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.472705 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert podName:76d17d44-d39f-4b04-9443-28a71704f3a8 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.472686869 +0000 UTC m=+45.157893621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-jq98v" (UID: "76d17d44-d39f-4b04-9443-28a71704f3a8") : secret "networking-console-plugin-cert" not found Jul 08 18:04:47.473209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.473124 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:04:47.473209 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:47.473174 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls podName:084a77c4-cf51-49c0-8cb0-1963bec63658 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.473157619 +0000 UTC m=+45.158364370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls") pod "dns-default-tj2cs" (UID: "084a77c4-cf51-49c0-8cb0-1963bec63658") : secret "dns-default-metrics-tls" not found Jul 08 18:04:47.611543 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.611335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:47.674542 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:47.674300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:48.070172 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:48.070102 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-46qjq" event={"ID":"319f6f85-dcd0-4060-89f5-8c32c347ba0d","Type":"ContainerStarted","Data":"20b38ef846b576a55f4b031d53b82e25f93b3e364bd2a07429040819b27a77a4"} Jul 08 18:04:48.098356 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:48.098282 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-46qjq" podStartSLOduration=8.516145304 podStartE2EDuration="43.098260013s" totalImagesPullingTime="34.582114709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:48.096747912 +0000 UTC m=+43.781954680" watchObservedRunningTime="2026-07-08 18:04:48.098260013 +0000 UTC m=+43.783466785" Jul 08 18:04:48.413380 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:48.413346 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:48.514463 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:48.514433 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:48.817297 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:48.817188 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:48.917420 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:48.917296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:49.289120 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.289028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:49.289571 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.289228 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:53.289207075 +0000 UTC m=+48.974413842 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : configmap references non-existent config key: service-ca.crt Jul 08 18:04:49.289571 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.289310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:49.289571 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.289343 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:49.289571 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.289460 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:04:49.289571 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.289473 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd9b879b-6xmsl: secret "image-registry-tls" not found Jul 08 18:04:49.289571 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.289515 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls podName:b6457a1b-e229-424b-a634-78951adc295f nodeName:}" failed. No retries permitted until 2026-07-08 18:04:53.289499014 +0000 UTC m=+48.974705763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls") pod "image-registry-85fd9b879b-6xmsl" (UID: "b6457a1b-e229-424b-a634-78951adc295f") : secret "image-registry-tls" not found Jul 08 18:04:49.289571 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.289575 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:04:49.289960 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.289605 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:53.289595926 +0000 UTC m=+48.974802699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : secret "router-metrics-certs-default" not found Jul 08 18:04:49.390083 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.389999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:49.390263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.390092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:49.390328 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.390313 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 18:04:49.390516 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.390381 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls podName:7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c nodeName:}" failed. No retries permitted until 2026-07-08 18:04:53.390361782 +0000 UTC m=+49.075568549 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-jsp8s" (UID: "7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c") : secret "samples-operator-tls" not found Jul 08 18:04:49.390516 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.390440 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:49.390516 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.390482 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls podName:3fddd715-cce6-4e4d-a611-adc89869fd3e nodeName:}" failed. No retries permitted until 2026-07-08 18:04:53.390469585 +0000 UTC m=+49.075676350 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-clrnw" (UID: "3fddd715-cce6-4e4d-a611-adc89869fd3e") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:49.490747 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.490702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:49.490939 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.490757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:49.490939 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.490870 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:04:49.490939 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.490937 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls podName:084a77c4-cf51-49c0-8cb0-1963bec63658 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:53.490918812 +0000 UTC m=+49.176125586 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls") pod "dns-default-tj2cs" (UID: "084a77c4-cf51-49c0-8cb0-1963bec63658") : secret "dns-default-metrics-tls" not found Jul 08 18:04:49.491142 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.490933 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 18:04:49.491142 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:49.490972 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert podName:76d17d44-d39f-4b04-9443-28a71704f3a8 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:53.490965884 +0000 UTC m=+49.176172631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-jq98v" (UID: "76d17d44-d39f-4b04-9443-28a71704f3a8") : secret "networking-console-plugin-cert" not found Jul 08 18:04:49.612968 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.612883 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:49.712752 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:49.712718 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:50.003668 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:50.003630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:50.069422 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:50.069385 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:50.820222 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:50.820174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:51.211953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:51.211920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:51.315508 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:51.315422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:52.323287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:52.323256 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:53.323244 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.323194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:53.323430 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.323316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:04:53.323430 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.323349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:04:53.323430 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.323376 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:01.323355016 +0000 UTC m=+57.008561776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : configmap references non-existent config key: service-ca.crt Jul 08 18:04:53.323853 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.323445 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:04:53.323853 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.323450 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:04:53.323853 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.323459 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd9b879b-6xmsl: secret "image-registry-tls" not found Jul 08 18:04:53.323853 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.323504 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:01.323488939 +0000 UTC m=+57.008695697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : secret "router-metrics-certs-default" not found Jul 08 18:04:53.323853 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.323520 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls podName:b6457a1b-e229-424b-a634-78951adc295f nodeName:}" failed. No retries permitted until 2026-07-08 18:05:01.323512472 +0000 UTC m=+57.008719219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls") pod "image-registry-85fd9b879b-6xmsl" (UID: "b6457a1b-e229-424b-a634-78951adc295f") : secret "image-registry-tls" not found Jul 08 18:04:53.424262 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.424228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:04:53.424371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.424318 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:04:53.424439 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.424390 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:53.424486 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.424475 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 18:04:53.424531 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.424508 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls podName:3fddd715-cce6-4e4d-a611-adc89869fd3e nodeName:}" failed. No retries permitted until 2026-07-08 18:05:01.424487739 +0000 UTC m=+57.109694489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-clrnw" (UID: "3fddd715-cce6-4e4d-a611-adc89869fd3e") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:04:53.424585 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.424545 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls podName:7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c nodeName:}" failed. No retries permitted until 2026-07-08 18:05:01.424526679 +0000 UTC m=+57.109733427 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-jsp8s" (UID: "7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c") : secret "samples-operator-tls" not found Jul 08 18:04:53.525293 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.525267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:04:53.525378 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.525314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:04:53.525446 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.525426 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:04:53.525510 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.525499 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls podName:084a77c4-cf51-49c0-8cb0-1963bec63658 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:01.525478037 +0000 UTC m=+57.210684785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls") pod "dns-default-tj2cs" (UID: "084a77c4-cf51-49c0-8cb0-1963bec63658") : secret "dns-default-metrics-tls" not found Jul 08 18:04:53.525571 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.525510 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 18:04:53.525624 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:53.525571 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert podName:76d17d44-d39f-4b04-9443-28a71704f3a8 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:01.525551028 +0000 UTC m=+57.210757797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-jq98v" (UID: "76d17d44-d39f-4b04-9443-28a71704f3a8") : secret "networking-console-plugin-cert" not found Jul 08 18:04:53.540281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:53.540250 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:54.091411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.091369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" event={"ID":"a5195670-b3c0-4736-a5dc-3faab67e0195","Type":"ContainerStarted","Data":"83c3171981723898e95dc222def9fa7b144c30de0181919d0eda547317ba3bc3"} Jul 08 18:04:54.093280 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.093251 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xsrcw" event={"ID":"3c398807-d5ee-4ab3-9cf6-94364606b1d6","Type":"ContainerStarted","Data":"0bf27770f332a9002488d7a2944d238bf61ac5c21cf391bf8b62c48b3dc87fe2"} Jul 08 18:04:54.094849 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.094813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" event={"ID":"7c57f641-7b79-42b9-a796-35eb35a22fc6","Type":"ContainerStarted","Data":"9e3fdeef84a37c75a0cbe4edfbda645356da0fab9077a6dfc43fc2a1e5fcb8ec"} Jul 08 18:04:54.096559 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.096536 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" event={"ID":"f0f85e84-3171-495f-a6f2-3d9def5ccc1c","Type":"ContainerStarted","Data":"5af8273f4f8aa157ba81f65d6f8b6c67f8fe3621c9bb7add9f7eca4599edea1a"} Jul 08 18:04:54.098279 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.098259 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/0.log" Jul 08 18:04:54.098370 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.098295 2580 generic.go:356] "Generic (PLEG): container finished" podID="6525a595-633b-454a-931a-5c2c55f5b8b4" containerID="c77d6f5e89e446e14d064ba5d5d99d7c2211ceabb745b5f879450955c65d5604" exitCode=255 Jul 08 18:04:54.098498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.098445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" event={"ID":"6525a595-633b-454a-931a-5c2c55f5b8b4","Type":"ContainerDied","Data":"c77d6f5e89e446e14d064ba5d5d99d7c2211ceabb745b5f879450955c65d5604"} Jul 08 18:04:54.098607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.098593 2580 scope.go:117] "RemoveContainer" containerID="c77d6f5e89e446e14d064ba5d5d99d7c2211ceabb745b5f879450955c65d5604" Jul 08 18:04:54.099966 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.099944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" event={"ID":"4f7d7b3b-c770-4330-ba51-218d3b049f6b","Type":"ContainerStarted","Data":"fe7644d8c20a1f3d63f060adb31742e242653a631f8d2ebb5319f551336fe85c"} Jul 08 18:04:54.101598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.101577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" event={"ID":"80a3f6d5-6f6d-42ef-861a-76a39a41a027","Type":"ContainerStarted","Data":"8eccde901d45eaa2bf5013c982713a04770c3015beda5aa62027aebbbbc6b202"} Jul 08 18:04:54.110498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.110449 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" podStartSLOduration=38.916443934 podStartE2EDuration="46.110431474s" totalImagesPullingTime="7.19398754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:54.109148911 +0000 UTC m=+49.794355683" watchObservedRunningTime="2026-07-08 18:04:54.110431474 +0000 UTC m=+49.795638246" Jul 08 18:04:54.146867 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.146796 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-57w86" podStartSLOduration=38.801583514 podStartE2EDuration="46.146774875s" totalImagesPullingTime="7.345191361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:54.128223011 +0000 UTC m=+49.813429780" watchObservedRunningTime="2026-07-08 18:04:54.146774875 +0000 UTC m=+49.831981646" Jul 08 18:04:54.165718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.165655 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" podStartSLOduration=38.94625856 podStartE2EDuration="46.165632901s" totalImagesPullingTime="7.219374341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:54.164683406 +0000 UTC m=+49.849890177" watchObservedRunningTime="2026-07-08 18:04:54.165632901 +0000 UTC m=+49.850839667" Jul 08 18:04:54.184605 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.184553 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-pnrdf" podStartSLOduration=39.223527622 podStartE2EDuration="46.184531978s" totalImagesPullingTime="6.961004356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:54.183358364 +0000 UTC m=+49.868565134" watchObservedRunningTime="2026-07-08 18:04:54.184531978 +0000 UTC m=+49.869738747" Jul 08 18:04:54.202252 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.202196 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" podStartSLOduration=39.208898109 podStartE2EDuration="46.202177707s" totalImagesPullingTime="6.993279598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:54.201560635 +0000 UTC m=+49.886767405" watchObservedRunningTime="2026-07-08 18:04:54.202177707 +0000 UTC m=+49.887384476" Jul 08 18:04:54.222158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.222097 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xsrcw" podStartSLOduration=2.232739731 podStartE2EDuration="9.222074712s" totalImagesPullingTime="6.989334981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:54.22062101 +0000 UTC m=+49.905827780" watchObservedRunningTime="2026-07-08 18:04:54.222074712 +0000 UTC m=+49.907281475" Jul 08 18:04:54.276377 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.276330 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:54.376116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:54.376014 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:55.107362 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.107337 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/1.log" Jul 08 18:04:55.107873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.107851 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/0.log" Jul 08 18:04:55.108014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.107892 2580 generic.go:356] "Generic (PLEG): container finished" podID="6525a595-633b-454a-931a-5c2c55f5b8b4" containerID="6217bac1180b7065ba97924d08b8f5e23a8beb812d147d404012a1cfc4c4ea8e" exitCode=255 Jul 08 18:04:55.108014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.107943 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" event={"ID":"6525a595-633b-454a-931a-5c2c55f5b8b4","Type":"ContainerDied","Data":"6217bac1180b7065ba97924d08b8f5e23a8beb812d147d404012a1cfc4c4ea8e"} Jul 08 18:04:55.108132 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.108061 2580 scope.go:117] "RemoveContainer" containerID="c77d6f5e89e446e14d064ba5d5d99d7c2211ceabb745b5f879450955c65d5604" Jul 08 18:04:55.108400 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.108324 2580 scope.go:117] "RemoveContainer" containerID="6217bac1180b7065ba97924d08b8f5e23a8beb812d147d404012a1cfc4c4ea8e" Jul 08 18:04:55.108543 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.108522 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4vx47_openshift-console-operator(6525a595-633b-454a-931a-5c2c55f5b8b4)\"" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" podUID="6525a595-633b-454a-931a-5c2c55f5b8b4" Jul 08 18:04:55.166289 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.166257 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hndg9"] Jul 08 18:04:55.202370 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.202332 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hndg9"] Jul 08 18:04:55.202521 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.202466 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.205114 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.205070 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 18:04:55.205255 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.205136 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tfxt7\"" Jul 08 18:04:55.205342 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.205328 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 18:04:55.233977 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.233942 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:55.243133 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.243028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.243296 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.243148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb83970e-f964-4be9-8882-4d1a8b25d3fa-data-volume\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.243296 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.243248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb83970e-f964-4be9-8882-4d1a8b25d3fa-crio-socket\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.243296 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.243277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjwlr\" (UniqueName: \"kubernetes.io/projected/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-api-access-hjwlr\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.243446 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.243329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.343768 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.343737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb83970e-f964-4be9-8882-4d1a8b25d3fa-data-volume\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.343951 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.343861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb83970e-f964-4be9-8882-4d1a8b25d3fa-crio-socket\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.343951 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.343890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjwlr\" (UniqueName: \"kubernetes.io/projected/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-api-access-hjwlr\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.343951 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.343937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.344119 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.343993 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.344119 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.344099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb83970e-f964-4be9-8882-4d1a8b25d3fa-crio-socket\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.344213 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.344121 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:04:55.344213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.344131 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb83970e-f964-4be9-8882-4d1a8b25d3fa-data-volume\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.344213 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.344200 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:04:55.844179143 +0000 UTC m=+51.529385911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : configmap "kube-rbac-proxy" not found Jul 08 18:04:55.344354 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.344223 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:04:55.344354 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.344278 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:04:55.844261812 +0000 UTC m=+51.529468573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : secret "insights-runtime-extractor-tls" not found Jul 08 18:04:55.344720 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.344696 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:55.355432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.355406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjwlr\" (UniqueName: \"kubernetes.io/projected/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-api-access-hjwlr\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.556706 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.556617 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:55.594731 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.594698 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl"] Jul 08 18:04:55.605152 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.605127 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" Jul 08 18:04:55.607562 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.607533 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 18:04:55.607562 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.607556 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:55.607733 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.607714 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4pr6d\"" Jul 08 18:04:55.608185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.608159 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl"] Jul 08 18:04:55.638879 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.638826 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:55.646571 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.646548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz46g\" (UniqueName: \"kubernetes.io/projected/59552106-b49a-4d61-8dd5-d6cea96444d1-kube-api-access-vz46g\") pod \"migrator-d57d58cc9-28nrl\" (UID: \"59552106-b49a-4d61-8dd5-d6cea96444d1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" Jul 08 18:04:55.695830 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.695797 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:55.747371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.747340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46g\" (UniqueName: \"kubernetes.io/projected/59552106-b49a-4d61-8dd5-d6cea96444d1-kube-api-access-vz46g\") pod \"migrator-d57d58cc9-28nrl\" (UID: \"59552106-b49a-4d61-8dd5-d6cea96444d1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" Jul 08 18:04:55.757642 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.757617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz46g\" (UniqueName: \"kubernetes.io/projected/59552106-b49a-4d61-8dd5-d6cea96444d1-kube-api-access-vz46g\") pod \"migrator-d57d58cc9-28nrl\" (UID: \"59552106-b49a-4d61-8dd5-d6cea96444d1\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" Jul 08 18:04:55.848597 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.848513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.848597 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.848580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:55.848800 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.848649 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:04:55.848800 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.848660 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:04:55.848800 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.848721 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:04:56.848703148 +0000 UTC m=+52.533909910 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : secret "insights-runtime-extractor-tls" not found Jul 08 18:04:55.848800 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:55.848767 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:04:56.848743229 +0000 UTC m=+52.533949977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : configmap "kube-rbac-proxy" not found Jul 08 18:04:55.914946 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.914910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:55.915625 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:55.915596 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" Jul 08 18:04:56.015779 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.015753 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:56.056558 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.056538 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl"] Jul 08 18:04:56.058747 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:56.058719 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59552106_b49a_4d61_8dd5_d6cea96444d1.slice/crio-7f5d0c651bd262136c56a5301d36bbeaa15a4acd2c44e6be93c07d58b4359140 WatchSource:0}: Error finding container 7f5d0c651bd262136c56a5301d36bbeaa15a4acd2c44e6be93c07d58b4359140: Status 404 returned error can't find the container with id 7f5d0c651bd262136c56a5301d36bbeaa15a4acd2c44e6be93c07d58b4359140 Jul 08 18:04:56.090009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.089978 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:56.090152 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.090027 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:04:56.111912 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.111815 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" event={"ID":"59552106-b49a-4d61-8dd5-d6cea96444d1","Type":"ContainerStarted","Data":"7f5d0c651bd262136c56a5301d36bbeaa15a4acd2c44e6be93c07d58b4359140"} Jul 08 18:04:56.113259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.113242 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/1.log" Jul 08 18:04:56.113545 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.113533 2580 scope.go:117] "RemoveContainer" containerID="6217bac1180b7065ba97924d08b8f5e23a8beb812d147d404012a1cfc4c4ea8e" Jul 08 18:04:56.113688 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:56.113671 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4vx47_openshift-console-operator(6525a595-633b-454a-931a-5c2c55f5b8b4)\"" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" podUID="6525a595-633b-454a-931a-5c2c55f5b8b4" Jul 08 18:04:56.147786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.147762 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lngwj_6d5114bf-8ccf-4e07-8295-8e0d447ca9e5/dns-node-resolver/0.log" Jul 08 18:04:56.265373 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.265335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:56.856422 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.856385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:56.857001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:56.856459 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:56.857001 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:56.856573 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:04:56.857001 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:56.856578 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:04:56.857001 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:56.856652 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:04:58.856632227 +0000 UTC m=+54.541838978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : secret "insights-runtime-extractor-tls" not found Jul 08 18:04:56.857001 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:56.856695 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:04:58.856678544 +0000 UTC m=+54.541885291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : configmap "kube-rbac-proxy" not found Jul 08 18:04:57.117941 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.117868 2580 scope.go:117] "RemoveContainer" containerID="6217bac1180b7065ba97924d08b8f5e23a8beb812d147d404012a1cfc4c4ea8e" Jul 08 18:04:57.118099 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:57.118074 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4vx47_openshift-console-operator(6525a595-633b-454a-931a-5c2c55f5b8b4)\"" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" podUID="6525a595-633b-454a-931a-5c2c55f5b8b4" Jul 08 18:04:57.348826 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.348800 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tw7m_5e04054b-06b5-4b90-9891-3e66ab6942c0/node-ca/0.log" Jul 08 18:04:57.524614 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.522533 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-7vk28"] Jul 08 18:04:57.545120 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.545087 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-7vk28"] Jul 08 18:04:57.545283 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.545205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.547633 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.547610 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 18:04:57.547782 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.547716 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 18:04:57.547782 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.547719 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 18:04:57.548669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.548651 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tm5fr\"" Jul 08 18:04:57.548798 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.548737 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 18:04:57.562609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.562574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:57.663717 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.663687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k22cn\" (UniqueName: \"kubernetes.io/projected/c4f4b392-5bfd-4623-b15b-eb8565fba493-kube-api-access-k22cn\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.663889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.663774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4f4b392-5bfd-4623-b15b-eb8565fba493-signing-cabundle\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.663889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.663797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4f4b392-5bfd-4623-b15b-eb8565fba493-signing-key\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.764545 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.764509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k22cn\" (UniqueName: \"kubernetes.io/projected/c4f4b392-5bfd-4623-b15b-eb8565fba493-kube-api-access-k22cn\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.764722 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.764626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4f4b392-5bfd-4623-b15b-eb8565fba493-signing-cabundle\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.764722 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.764661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4f4b392-5bfd-4623-b15b-eb8565fba493-signing-key\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.765376 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.765350 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4f4b392-5bfd-4623-b15b-eb8565fba493-signing-cabundle\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.767253 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.767230 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4f4b392-5bfd-4623-b15b-eb8565fba493-signing-key\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.773189 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.773165 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k22cn\" (UniqueName: \"kubernetes.io/projected/c4f4b392-5bfd-4623-b15b-eb8565fba493-kube-api-access-k22cn\") pod \"service-ca-758f599c5-7vk28\" (UID: \"c4f4b392-5bfd-4623-b15b-eb8565fba493\") " pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.854856 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.854764 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-7vk28" Jul 08 18:04:57.948786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:57.948763 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xsrcw_3c398807-d5ee-4ab3-9cf6-94364606b1d6/serve-healthcheck-canary/0.log" Jul 08 18:04:58.066289 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:58.066253 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-7vk28"] Jul 08 18:04:58.069782 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:04:58.069754 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f4b392_5bfd_4623_b15b_eb8565fba493.slice/crio-bc5cd1153a1bf6b83503239f1c430ea4c441c2fab37c3ac2198d38bec024d5e6 WatchSource:0}: Error finding container bc5cd1153a1bf6b83503239f1c430ea4c441c2fab37c3ac2198d38bec024d5e6: Status 404 returned error can't find the container with id bc5cd1153a1bf6b83503239f1c430ea4c441c2fab37c3ac2198d38bec024d5e6 Jul 08 18:04:58.123349 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:58.123310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-7vk28" event={"ID":"c4f4b392-5bfd-4623-b15b-eb8565fba493","Type":"ContainerStarted","Data":"bc5cd1153a1bf6b83503239f1c430ea4c441c2fab37c3ac2198d38bec024d5e6"} Jul 08 18:04:58.624655 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:58.624622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:58.873113 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:58.873077 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:58.873363 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:58.873152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:04:58.873363 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:58.873224 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:04:58.873363 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:58.873301 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:05:02.873280147 +0000 UTC m=+58.558486893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : secret "insights-runtime-extractor-tls" not found Jul 08 18:04:58.873363 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:58.873313 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:04:58.873581 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:04:58.873372 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:05:02.8733546 +0000 UTC m=+58.558561368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : configmap "kube-rbac-proxy" not found Jul 08 18:04:59.050199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.050061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:59.129163 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.129035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-7vk28" event={"ID":"c4f4b392-5bfd-4623-b15b-eb8565fba493","Type":"ContainerStarted","Data":"4217056b9fda068a2b88a051f31cb4f261f23290d22a0fe61a0ecb22a74e2f6b"} Jul 08 18:04:59.131406 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.131377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" event={"ID":"59552106-b49a-4d61-8dd5-d6cea96444d1","Type":"ContainerStarted","Data":"988292b674cdec9b564fbe4e3444dad3f6de902217225db0c4a45ef28253ea8b"} Jul 08 18:04:59.131539 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.131425 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" event={"ID":"59552106-b49a-4d61-8dd5-d6cea96444d1","Type":"ContainerStarted","Data":"15c7fd18a1c8b415299549c45863ad74c20edcad2acc8a4c8e612afe6beaeb90"} Jul 08 18:04:59.150540 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.150484 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-7vk28" podStartSLOduration=2.15046464 podStartE2EDuration="2.15046464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:59.148567128 +0000 UTC m=+54.833773899" watchObservedRunningTime="2026-07-08 18:04:59.15046464 +0000 UTC m=+54.835671409" Jul 08 18:04:59.172945 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.172884 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-28nrl" podStartSLOduration=1.882865995 podStartE2EDuration="4.172863674s" totalImagesPullingTime="2.289997679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:04:59.171259199 +0000 UTC m=+54.856465967" watchObservedRunningTime="2026-07-08 18:04:59.172863674 +0000 UTC m=+54.858070445" Jul 08 18:04:59.177582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.177464 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:59.246073 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.246042 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:59.345159 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.345123 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:59.446890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.446849 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:59.922743 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.922705 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:04:59.982254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:04:59.982219 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:00.111279 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:00.111244 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:00.212438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:00.212348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:00.742418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:00.742369 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:01.397976 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.397947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.398030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.398069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.398125 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:17.398101862 +0000 UTC m=+73.083308609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : configmap references non-existent config key: service-ca.crt Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.398204 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.398207 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.398275 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs podName:6c74951d-5706-4a46-9163-5b9d3dc67042 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:17.398258941 +0000 UTC m=+73.083465688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs") pod "router-default-6d459b6969-8sg97" (UID: "6c74951d-5706-4a46-9163-5b9d3dc67042") : secret "router-metrics-certs-default" not found Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.398215 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd9b879b-6xmsl: secret "image-registry-tls" not found Jul 08 18:05:01.398422 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.398319 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls podName:b6457a1b-e229-424b-a634-78951adc295f nodeName:}" failed. No retries permitted until 2026-07-08 18:05:17.398309772 +0000 UTC m=+73.083516521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls") pod "image-registry-85fd9b879b-6xmsl" (UID: "b6457a1b-e229-424b-a634-78951adc295f") : secret "image-registry-tls" not found Jul 08 18:05:01.498626 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.498584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:05:01.498809 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.498670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:05:01.498809 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.498739 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 18:05:01.498931 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.498811 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls podName:3fddd715-cce6-4e4d-a611-adc89869fd3e nodeName:}" failed. No retries permitted until 2026-07-08 18:05:17.498794354 +0000 UTC m=+73.184001101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-clrnw" (UID: "3fddd715-cce6-4e4d-a611-adc89869fd3e") : secret "cluster-monitoring-operator-tls" not found Jul 08 18:05:01.501349 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.501321 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-jsp8s\" (UID: \"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:05:01.532633 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.532602 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:01.573634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.573604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" Jul 08 18:05:01.599351 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.599326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:05:01.599481 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.599358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:05:01.599521 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.599492 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:05:01.599581 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.599569 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls podName:084a77c4-cf51-49c0-8cb0-1963bec63658 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:17.599553227 +0000 UTC m=+73.284759977 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls") pod "dns-default-tj2cs" (UID: "084a77c4-cf51-49c0-8cb0-1963bec63658") : secret "dns-default-metrics-tls" not found Jul 08 18:05:01.599581 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.599492 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 18:05:01.599657 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:01.599612 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert podName:76d17d44-d39f-4b04-9443-28a71704f3a8 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:17.599601077 +0000 UTC m=+73.284807838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-jq98v" (UID: "76d17d44-d39f-4b04-9443-28a71704f3a8") : secret "networking-console-plugin-cert" not found Jul 08 18:05:01.707744 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.707719 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s"] Jul 08 18:05:01.854518 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.854486 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:01.961946 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:01.961862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:02.017923 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.017886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:02.133776 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.133739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:02.142372 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.142346 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" event={"ID":"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c","Type":"ContainerStarted","Data":"fd3e69400fb5b2a4e096b41ef9e8203737a7676fa46cff8651bc1e0ff2c877ce"} Jul 08 18:05:02.235372 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.235289 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:02.337278 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.337240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:02.546560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.546375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:02.741199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.741158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:02.914172 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.914139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:05:02.914356 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:02.914182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:05:02.914356 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:02.914284 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:05:02.914356 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:02.914333 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:05:02.914514 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:02.914360 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:05:10.914343311 +0000 UTC m=+66.599550059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : secret "insights-runtime-extractor-tls" not found Jul 08 18:05:02.914514 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:02.914394 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm podName:bb83970e-f964-4be9-8882-4d1a8b25d3fa nodeName:}" failed. No retries permitted until 2026-07-08 18:05:10.914373939 +0000 UTC m=+66.599580700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hndg9" (UID: "bb83970e-f964-4be9-8882-4d1a8b25d3fa") : configmap "kube-rbac-proxy" not found Jul 08 18:05:03.106226 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:03.106178 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:03.342438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:03.342346 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:03.962924 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:03.962889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:04.063605 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:04.063570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:04.337331 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:04.337247 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:04.790385 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:04.790348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:04.856799 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:04.856683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:05.145615 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:05.145589 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:05.155062 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:05.155026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" event={"ID":"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c","Type":"ContainerStarted","Data":"9a87c49941ee20ea5397aa858214b6e9aa8b1cd10309f808b27c6518847004f4"} Jul 08 18:05:05.155191 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:05.155086 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" event={"ID":"7fa5d9c3-8e83-40a1-8202-cd68d47c0f6c","Type":"ContainerStarted","Data":"2782619ebf9eca0deec685f30321ace6591e2eeae3850eafe0f1f903006fa324"} Jul 08 18:05:05.175009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:05.174956 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-jsp8s" podStartSLOduration=54.437139585 podStartE2EDuration="57.17493619s" totalImagesPullingTime="2.737796605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:05.17310441 +0000 UTC m=+60.858311182" watchObservedRunningTime="2026-07-08 18:05:05.17493619 +0000 UTC m=+60.860142960" Jul 08 18:05:05.304619 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:05.304582 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:05.937662 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:05.937628 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:06.141527 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:06.141488 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:06.242075 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:06.241992 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:06.539650 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:06.539572 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:06.742241 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:06.742206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:07.335817 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:07.335788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:07.950371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:07.950333 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:08.735609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:08.735563 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:08.837905 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:08.837866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:08.951333 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:08.951296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:09.536339 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:09.536300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:09.863579 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:09.863545 2580 scope.go:117] "RemoveContainer" containerID="6217bac1180b7065ba97924d08b8f5e23a8beb812d147d404012a1cfc4c4ea8e" Jul 08 18:05:09.912564 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:09.912536 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:10.172011 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.171982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:05:10.172405 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.172390 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/1.log" Jul 08 18:05:10.172458 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.172422 2580 generic.go:356] "Generic (PLEG): container finished" podID="6525a595-633b-454a-931a-5c2c55f5b8b4" containerID="ff9e119d4b5a0f1d093cd805a1559b58a43c1d648aad1094e52e0504dc252b4c" exitCode=255 Jul 08 18:05:10.172493 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.172464 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" event={"ID":"6525a595-633b-454a-931a-5c2c55f5b8b4","Type":"ContainerDied","Data":"ff9e119d4b5a0f1d093cd805a1559b58a43c1d648aad1094e52e0504dc252b4c"} Jul 08 18:05:10.172526 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.172507 2580 scope.go:117] "RemoveContainer" containerID="6217bac1180b7065ba97924d08b8f5e23a8beb812d147d404012a1cfc4c4ea8e" Jul 08 18:05:10.172848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.172811 2580 scope.go:117] "RemoveContainer" containerID="ff9e119d4b5a0f1d093cd805a1559b58a43c1d648aad1094e52e0504dc252b4c" Jul 08 18:05:10.173037 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:10.173016 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-4vx47_openshift-console-operator(6525a595-633b-454a-931a-5c2c55f5b8b4)\"" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" podUID="6525a595-633b-454a-931a-5c2c55f5b8b4" Jul 08 18:05:10.556946 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.556820 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:10.688300 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.688267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:05:10.688450 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.688353 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:05:10.691572 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.691554 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:05:10.691661 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.691601 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:05:10.700650 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.700630 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ad1bba-365f-43c5-b06f-13af24af75c3-metrics-certs\") pod \"network-metrics-daemon-xvmqm\" (UID: \"21ad1bba-365f-43c5-b06f-13af24af75c3\") " pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:05:10.700763 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.700747 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ee9becb-c169-4398-b46e-7daa536542a4-original-pull-secret\") pod \"global-pull-secret-syncer-znlq7\" (UID: \"7ee9becb-c169-4398-b46e-7daa536542a4\") " pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:05:10.789426 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.789393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:05:10.791800 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.791776 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79lgv\" (UniqueName: \"kubernetes.io/projected/19db2ce5-00a8-43fa-9927-c747b6fb9d9e-kube-api-access-79lgv\") pod \"network-check-target-gkkkh\" (UID: \"19db2ce5-00a8-43fa-9927-c747b6fb9d9e\") " pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:05:10.888693 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.888672 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d9s8r\"" Jul 08 18:05:10.893040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.893018 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8pfq\"" Jul 08 18:05:10.893146 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.893051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-znlq7" Jul 08 18:05:10.896453 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.896439 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:05:10.901125 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.901102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvmqm" Jul 08 18:05:10.991444 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.990584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:05:10.991444 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.990645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:05:10.991444 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.991345 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb83970e-f964-4be9-8882-4d1a8b25d3fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:05:10.993995 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:10.993956 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb83970e-f964-4be9-8882-4d1a8b25d3fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hndg9\" (UID: \"bb83970e-f964-4be9-8882-4d1a8b25d3fa\") " pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:05:11.033200 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.033179 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dpdlj" Jul 08 18:05:11.039624 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.039601 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-znlq7"] Jul 08 18:05:11.043555 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:11.043535 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee9becb_c169_4398_b46e_7daa536542a4.slice/crio-e9f87ad6e1341495656c342b84c4fe3fec8745df064da069d89b5824fbd21de9 WatchSource:0}: Error finding container e9f87ad6e1341495656c342b84c4fe3fec8745df064da069d89b5824fbd21de9: Status 404 returned error can't find the container with id e9f87ad6e1341495656c342b84c4fe3fec8745df064da069d89b5824fbd21de9 Jul 08 18:05:11.115936 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.115913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tfxt7\"" Jul 08 18:05:11.123443 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.123426 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hndg9" Jul 08 18:05:11.178968 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.178937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-znlq7" event={"ID":"7ee9becb-c169-4398-b46e-7daa536542a4","Type":"ContainerStarted","Data":"e9f87ad6e1341495656c342b84c4fe3fec8745df064da069d89b5824fbd21de9"} Jul 08 18:05:11.180323 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.180305 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:05:11.261376 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.261353 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hndg9"] Jul 08 18:05:11.263176 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:11.263153 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb83970e_f964_4be9_8882_4d1a8b25d3fa.slice/crio-4a9c8f028b87532700dbc56e05d0bc96042066542b6ec18f77655248ade0910e WatchSource:0}: Error finding container 4a9c8f028b87532700dbc56e05d0bc96042066542b6ec18f77655248ade0910e: Status 404 returned error can't find the container with id 4a9c8f028b87532700dbc56e05d0bc96042066542b6ec18f77655248ade0910e Jul 08 18:05:11.279801 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.279778 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gkkkh"] Jul 08 18:05:11.282790 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:11.282769 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19db2ce5_00a8_43fa_9927_c747b6fb9d9e.slice/crio-bf6001c75913abbe3c830b21762a7c891bfbc2b4f9c3324b77a99b96baeb5081 WatchSource:0}: Error finding container bf6001c75913abbe3c830b21762a7c891bfbc2b4f9c3324b77a99b96baeb5081: Status 404 returned error can't find the container with id bf6001c75913abbe3c830b21762a7c891bfbc2b4f9c3324b77a99b96baeb5081 Jul 08 18:05:11.285381 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:11.285359 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xvmqm"] Jul 08 18:05:11.287336 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:11.287316 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21ad1bba_365f_43c5_b06f_13af24af75c3.slice/crio-3266eeec55c2dfd117c3889bbbb4a1b33f441b1d17a684efe113456370d9fa41 WatchSource:0}: Error finding container 3266eeec55c2dfd117c3889bbbb4a1b33f441b1d17a684efe113456370d9fa41: Status 404 returned error can't find the container with id 3266eeec55c2dfd117c3889bbbb4a1b33f441b1d17a684efe113456370d9fa41 Jul 08 18:05:12.188564 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:12.188461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hndg9" event={"ID":"bb83970e-f964-4be9-8882-4d1a8b25d3fa","Type":"ContainerStarted","Data":"4a9c8f028b87532700dbc56e05d0bc96042066542b6ec18f77655248ade0910e"} Jul 08 18:05:12.191264 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:12.191150 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gkkkh" event={"ID":"19db2ce5-00a8-43fa-9927-c747b6fb9d9e","Type":"ContainerStarted","Data":"e6de4d4c70e14172ed87d1c95d4aa3cfb0e9b9306fc5ba6a9bda33ce4445aaba"} Jul 08 18:05:12.191264 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:12.191202 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gkkkh" event={"ID":"19db2ce5-00a8-43fa-9927-c747b6fb9d9e","Type":"ContainerStarted","Data":"bf6001c75913abbe3c830b21762a7c891bfbc2b4f9c3324b77a99b96baeb5081"} Jul 08 18:05:12.191264 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:12.191250 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:05:12.192938 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:12.192914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvmqm" event={"ID":"21ad1bba-365f-43c5-b06f-13af24af75c3","Type":"ContainerStarted","Data":"3266eeec55c2dfd117c3889bbbb4a1b33f441b1d17a684efe113456370d9fa41"} Jul 08 18:05:12.221997 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:12.221943 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gkkkh" podStartSLOduration=67.221923706 podStartE2EDuration="1m7.221923706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:12.220273802 +0000 UTC m=+67.905480571" watchObservedRunningTime="2026-07-08 18:05:12.221923706 +0000 UTC m=+67.907130476" Jul 08 18:05:14.203677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:14.203631 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvmqm" event={"ID":"21ad1bba-365f-43c5-b06f-13af24af75c3","Type":"ContainerStarted","Data":"babf123e06a502968df3a5a793706a67155ee148a86ac2e65549fe4cbf6a154e"} Jul 08 18:05:14.204149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:14.203699 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvmqm" event={"ID":"21ad1bba-365f-43c5-b06f-13af24af75c3","Type":"ContainerStarted","Data":"3c2392b045e9c6e250df178c3937eb42ea19eca26ca900fee27e651522549b7a"} Jul 08 18:05:14.231569 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:14.231514 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xvmqm" podStartSLOduration=66.224835542 podStartE2EDuration="1m8.231496203s" totalImagesPullingTime="2.006660661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:14.229774357 +0000 UTC m=+69.914981128" watchObservedRunningTime="2026-07-08 18:05:14.231496203 +0000 UTC m=+69.916702973" Jul 08 18:05:15.125978 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:15.125892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:15.195516 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:15.195464 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:16.089526 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:16.089495 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:05:16.090002 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:16.089587 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:05:16.090079 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:16.090009 2580 scope.go:117] "RemoveContainer" containerID="ff9e119d4b5a0f1d093cd805a1559b58a43c1d648aad1094e52e0504dc252b4c" Jul 08 18:05:16.090217 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:05:16.090189 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-4vx47_openshift-console-operator(6525a595-633b-454a-931a-5c2c55f5b8b4)\"" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" podUID="6525a595-633b-454a-931a-5c2c55f5b8b4" Jul 08 18:05:17.219523 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.219496 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hndg9" event={"ID":"bb83970e-f964-4be9-8882-4d1a8b25d3fa","Type":"ContainerStarted","Data":"1912138ec238cb1690354c0aaade421fad9490f30fb9070c89932af7c71961b0"} Jul 08 18:05:17.451432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.451401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:05:17.451688 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.451497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:17.451688 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.451617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:17.452116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.452089 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c74951d-5706-4a46-9163-5b9d3dc67042-service-ca-bundle\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:17.453773 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.453741 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"image-registry-85fd9b879b-6xmsl\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:05:17.453869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.453783 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c74951d-5706-4a46-9163-5b9d3dc67042-metrics-certs\") pod \"router-default-6d459b6969-8sg97\" (UID: \"6c74951d-5706-4a46-9163-5b9d3dc67042\") " pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:17.553027 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.552994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:05:17.555241 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.555220 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3fddd715-cce6-4e4d-a611-adc89869fd3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-clrnw\" (UID: \"3fddd715-cce6-4e4d-a611-adc89869fd3e\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:05:17.596955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.596920 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mjh89\"" Jul 08 18:05:17.604557 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.604538 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" Jul 08 18:05:17.654163 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.654117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:05:17.654328 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.654174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:05:17.656468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.656445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/084a77c4-cf51-49c0-8cb0-1963bec63658-metrics-tls\") pod \"dns-default-tj2cs\" (UID: \"084a77c4-cf51-49c0-8cb0-1963bec63658\") " pod="openshift-dns/dns-default-tj2cs" Jul 08 18:05:17.656626 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.656583 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76d17d44-d39f-4b04-9443-28a71704f3a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jq98v\" (UID: \"76d17d44-d39f-4b04-9443-28a71704f3a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:05:17.723845 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.723170 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hgp88\"" Jul 08 18:05:17.725318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.725215 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:17.735522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.735491 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9qxwl\"" Jul 08 18:05:17.743634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.743609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:05:17.765813 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.765785 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw"] Jul 08 18:05:17.769877 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:17.769784 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fddd715_cce6_4e4d_a611_adc89869fd3e.slice/crio-a4e3ff9e2224301f4da8068876a0e47cc590528296cde66193cd22c3e2d61f36 WatchSource:0}: Error finding container a4e3ff9e2224301f4da8068876a0e47cc590528296cde66193cd22c3e2d61f36: Status 404 returned error can't find the container with id a4e3ff9e2224301f4da8068876a0e47cc590528296cde66193cd22c3e2d61f36 Jul 08 18:05:17.903673 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.903650 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85fd9b879b-6xmsl"] Jul 08 18:05:17.903765 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.903748 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jfbrc\"" Jul 08 18:05:17.906998 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:17.906970 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6457a1b_e229_424b_a634_78951adc295f.slice/crio-7b28edb61e0c83c9d552cdefff2c129ef309f2fdb1eee71c3d3c867ac803b468 WatchSource:0}: Error finding container 7b28edb61e0c83c9d552cdefff2c129ef309f2fdb1eee71c3d3c867ac803b468: Status 404 returned error can't find the container with id 7b28edb61e0c83c9d552cdefff2c129ef309f2fdb1eee71c3d3c867ac803b468 Jul 08 18:05:17.911053 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.911032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" Jul 08 18:05:17.921304 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.921285 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmswz\"" Jul 08 18:05:17.924416 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.924395 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d459b6969-8sg97"] Jul 08 18:05:17.926941 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:17.926915 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c74951d_5706_4a46_9163_5b9d3dc67042.slice/crio-ca0769ab02cd7a762bc8c4b799375a50011bd02794e9834f4000e66883818406 WatchSource:0}: Error finding container ca0769ab02cd7a762bc8c4b799375a50011bd02794e9834f4000e66883818406: Status 404 returned error can't find the container with id ca0769ab02cd7a762bc8c4b799375a50011bd02794e9834f4000e66883818406 Jul 08 18:05:17.928727 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:17.928713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tj2cs" Jul 08 18:05:18.090447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.090405 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-jq98v"] Jul 08 18:05:18.093372 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:18.093336 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d17d44_d39f_4b04_9443_28a71704f3a8.slice/crio-ba09e055034e2f6bf20628657b6f88a6ddd9dc9f52d980bfd4b9e745ede647e4 WatchSource:0}: Error finding container ba09e055034e2f6bf20628657b6f88a6ddd9dc9f52d980bfd4b9e745ede647e4: Status 404 returned error can't find the container with id ba09e055034e2f6bf20628657b6f88a6ddd9dc9f52d980bfd4b9e745ede647e4 Jul 08 18:05:18.116014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.115990 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tj2cs"] Jul 08 18:05:18.118174 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:18.118139 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod084a77c4_cf51_49c0_8cb0_1963bec63658.slice/crio-a77ced398b6bdf9539bc2c83f12e2c1f322a7310a95ccc8e5bfc24da0754925e WatchSource:0}: Error finding container a77ced398b6bdf9539bc2c83f12e2c1f322a7310a95ccc8e5bfc24da0754925e: Status 404 returned error can't find the container with id a77ced398b6bdf9539bc2c83f12e2c1f322a7310a95ccc8e5bfc24da0754925e Jul 08 18:05:18.229594 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.229507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-znlq7" event={"ID":"7ee9becb-c169-4398-b46e-7daa536542a4","Type":"ContainerStarted","Data":"0a5e9522b46ba2f04f5ad27dfdd2e3eab60feab532d03645ff5b9b6c633db884"} Jul 08 18:05:18.231195 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.231169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d459b6969-8sg97" event={"ID":"6c74951d-5706-4a46-9163-5b9d3dc67042","Type":"ContainerStarted","Data":"108bbb90e3c0ee02e1a8dcdbb40d6dacbdf40516bccd6fd022250dd73e641bd9"} Jul 08 18:05:18.231321 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.231213 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d459b6969-8sg97" event={"ID":"6c74951d-5706-4a46-9163-5b9d3dc67042","Type":"ContainerStarted","Data":"ca0769ab02cd7a762bc8c4b799375a50011bd02794e9834f4000e66883818406"} Jul 08 18:05:18.232302 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.232271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tj2cs" event={"ID":"084a77c4-cf51-49c0-8cb0-1963bec63658","Type":"ContainerStarted","Data":"a77ced398b6bdf9539bc2c83f12e2c1f322a7310a95ccc8e5bfc24da0754925e"} Jul 08 18:05:18.233395 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.233368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" event={"ID":"76d17d44-d39f-4b04-9443-28a71704f3a8","Type":"ContainerStarted","Data":"ba09e055034e2f6bf20628657b6f88a6ddd9dc9f52d980bfd4b9e745ede647e4"} Jul 08 18:05:18.235039 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.235020 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hndg9" event={"ID":"bb83970e-f964-4be9-8882-4d1a8b25d3fa","Type":"ContainerStarted","Data":"244d4f9db915886a1d14eaa496cc36d2909aa8ffa53a67bfa8fdfcf23982f19b"} Jul 08 18:05:18.236490 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.236461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" event={"ID":"b6457a1b-e229-424b-a634-78951adc295f","Type":"ContainerStarted","Data":"0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4"} Jul 08 18:05:18.236590 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.236498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" event={"ID":"b6457a1b-e229-424b-a634-78951adc295f","Type":"ContainerStarted","Data":"7b28edb61e0c83c9d552cdefff2c129ef309f2fdb1eee71c3d3c867ac803b468"} Jul 08 18:05:18.236590 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.236524 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:05:18.237553 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.237533 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" event={"ID":"3fddd715-cce6-4e4d-a611-adc89869fd3e","Type":"ContainerStarted","Data":"a4e3ff9e2224301f4da8068876a0e47cc590528296cde66193cd22c3e2d61f36"} Jul 08 18:05:18.272149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.272095 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6d459b6969-8sg97" podStartSLOduration=70.272073261 podStartE2EDuration="1m10.272073261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:18.271573528 +0000 UTC m=+73.956780339" watchObservedRunningTime="2026-07-08 18:05:18.272073261 +0000 UTC m=+73.957280032" Jul 08 18:05:18.273651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.273605 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-znlq7" podStartSLOduration=66.199503874 podStartE2EDuration="1m12.273589527s" totalImagesPullingTime="6.074085653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:18.250281833 +0000 UTC m=+73.935488605" watchObservedRunningTime="2026-07-08 18:05:18.273589527 +0000 UTC m=+73.958796297" Jul 08 18:05:18.297458 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.297370 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" podStartSLOduration=72.297351087 podStartE2EDuration="1m12.297351087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:18.295473996 +0000 UTC m=+73.980680765" watchObservedRunningTime="2026-07-08 18:05:18.297351087 +0000 UTC m=+73.982557857" Jul 08 18:05:18.725823 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.725785 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:18.728700 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:18.728677 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:19.242805 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.242774 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:19.244548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.244348 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d459b6969-8sg97" Jul 08 18:05:19.277632 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.277601 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d"] Jul 08 18:05:19.303541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.301988 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp"] Jul 08 18:05:19.303541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.303413 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.309341 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.308290 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 18:05:19.309505 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.309381 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 18:05:19.310206 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.309606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 18:05:19.310206 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.309810 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 18:05:19.310206 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.309947 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 18:05:19.310206 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.310102 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 18:05:19.310432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.310254 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 18:05:19.317351 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.317275 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r"] Jul 08 18:05:19.317687 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.317522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.320908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.320720 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-429h7\"" Jul 08 18:05:19.320908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.320856 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 18:05:19.339511 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.339488 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp"] Jul 08 18:05:19.339685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.339524 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d"] Jul 08 18:05:19.339685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.339545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r"] Jul 08 18:05:19.339685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.339583 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.342936 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.342916 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 08 18:05:19.465731 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.465697 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:19.471920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.471769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.471920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.471818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf0919cf-dfb8-4f25-8dc2-678163025d26-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86b95f8b76-5wjvp\" (UID: \"bf0919cf-dfb8-4f25-8dc2-678163025d26\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.471920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.471870 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwr7g\" (UniqueName: \"kubernetes.io/projected/d2d495b3-193b-4caf-9329-b9645cebbd40-kube-api-access-qwr7g\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.472185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.471952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2d495b3-193b-4caf-9329-b9645cebbd40-tmp\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.472185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.472028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/07539c45-3389-487c-a42c-b9bb48b09de1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.472185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.472078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6wbk\" (UniqueName: \"kubernetes.io/projected/bf0919cf-dfb8-4f25-8dc2-678163025d26-kube-api-access-h6wbk\") pod \"managed-serviceaccount-addon-agent-86b95f8b76-5wjvp\" (UID: \"bf0919cf-dfb8-4f25-8dc2-678163025d26\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.472185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.472116 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-hub\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.472185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.472139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.472185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.472167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d2d495b3-193b-4caf-9329-b9645cebbd40-klusterlet-config\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.472473 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.472195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqtrr\" (UniqueName: \"kubernetes.io/projected/07539c45-3389-487c-a42c-b9bb48b09de1-kube-api-access-fqtrr\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.472473 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.472238 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-ca\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.573589 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.573510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6wbk\" (UniqueName: \"kubernetes.io/projected/bf0919cf-dfb8-4f25-8dc2-678163025d26-kube-api-access-h6wbk\") pod \"managed-serviceaccount-addon-agent-86b95f8b76-5wjvp\" (UID: \"bf0919cf-dfb8-4f25-8dc2-678163025d26\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.573589 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.573570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-hub\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.573882 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.573597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.573882 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.573622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d2d495b3-193b-4caf-9329-b9645cebbd40-klusterlet-config\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.573882 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.573646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqtrr\" (UniqueName: \"kubernetes.io/projected/07539c45-3389-487c-a42c-b9bb48b09de1-kube-api-access-fqtrr\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.574077 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.574056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-ca\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.574142 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.574123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.574198 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.574165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf0919cf-dfb8-4f25-8dc2-678163025d26-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86b95f8b76-5wjvp\" (UID: \"bf0919cf-dfb8-4f25-8dc2-678163025d26\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.574251 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.574226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwr7g\" (UniqueName: \"kubernetes.io/projected/d2d495b3-193b-4caf-9329-b9645cebbd40-kube-api-access-qwr7g\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.574306 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.574270 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2d495b3-193b-4caf-9329-b9645cebbd40-tmp\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.574352 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.574308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/07539c45-3389-487c-a42c-b9bb48b09de1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.575517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.575052 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/07539c45-3389-487c-a42c-b9bb48b09de1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.575517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.575479 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2d495b3-193b-4caf-9329-b9645cebbd40-tmp\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.577398 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.577370 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-ca\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.577652 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.577557 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-hub\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.577652 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.577605 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d2d495b3-193b-4caf-9329-b9645cebbd40-klusterlet-config\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.578130 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.578097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.578818 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.578794 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bf0919cf-dfb8-4f25-8dc2-678163025d26-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86b95f8b76-5wjvp\" (UID: \"bf0919cf-dfb8-4f25-8dc2-678163025d26\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.579330 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.579297 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/07539c45-3389-487c-a42c-b9bb48b09de1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.589298 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.588888 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:19.594201 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.593023 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwr7g\" (UniqueName: \"kubernetes.io/projected/d2d495b3-193b-4caf-9329-b9645cebbd40-kube-api-access-qwr7g\") pod \"klusterlet-addon-workmgr-766697ddd7-swf5r\" (UID: \"d2d495b3-193b-4caf-9329-b9645cebbd40\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.594201 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.594092 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqtrr\" (UniqueName: \"kubernetes.io/projected/07539c45-3389-487c-a42c-b9bb48b09de1-kube-api-access-fqtrr\") pod \"cluster-proxy-proxy-agent-57b67d4c6c-bkc5d\" (UID: \"07539c45-3389-487c-a42c-b9bb48b09de1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.594887 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.594868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6wbk\" (UniqueName: \"kubernetes.io/projected/bf0919cf-dfb8-4f25-8dc2-678163025d26-kube-api-access-h6wbk\") pod \"managed-serviceaccount-addon-agent-86b95f8b76-5wjvp\" (UID: \"bf0919cf-dfb8-4f25-8dc2-678163025d26\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.621891 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.621816 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" Jul 08 18:05:19.633055 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.632993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" Jul 08 18:05:19.654763 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.654740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:19.679308 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.679271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:19.790814 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:19.790777 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:20.094471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:20.094418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:20.205996 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:20.205854 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:20.297121 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:20.297077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:20.416767 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:20.416701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:21.133756 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:21.133716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:22.806571 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:22.806445 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp"] Jul 08 18:05:22.884288 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:22.884163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d"] Jul 08 18:05:22.908898 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:22.908877 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r"] Jul 08 18:05:22.912293 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:22.912263 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf0919cf_dfb8_4f25_8dc2_678163025d26.slice/crio-dd3a9b99c6691344b361d8d4603a1c4a8326827e705d6da96e4a5501f94b292a WatchSource:0}: Error finding container dd3a9b99c6691344b361d8d4603a1c4a8326827e705d6da96e4a5501f94b292a: Status 404 returned error can't find the container with id dd3a9b99c6691344b361d8d4603a1c4a8326827e705d6da96e4a5501f94b292a Jul 08 18:05:22.912742 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:22.912719 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07539c45_3389_487c_a42c_b9bb48b09de1.slice/crio-1b455f42eb13100d2f4b16397f01cf96ff20f10081e014c16674e39cc3869f96 WatchSource:0}: Error finding container 1b455f42eb13100d2f4b16397f01cf96ff20f10081e014c16674e39cc3869f96: Status 404 returned error can't find the container with id 1b455f42eb13100d2f4b16397f01cf96ff20f10081e014c16674e39cc3869f96 Jul 08 18:05:22.913850 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:22.913812 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d495b3_193b_4caf_9329_b9645cebbd40.slice/crio-24675ad19f8c3de405ed4431cd18557b96ca623d5b07e918dda0ffc192f8707f WatchSource:0}: Error finding container 24675ad19f8c3de405ed4431cd18557b96ca623d5b07e918dda0ffc192f8707f: Status 404 returned error can't find the container with id 24675ad19f8c3de405ed4431cd18557b96ca623d5b07e918dda0ffc192f8707f Jul 08 18:05:23.265639 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.265555 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" event={"ID":"76d17d44-d39f-4b04-9443-28a71704f3a8","Type":"ContainerStarted","Data":"6c553e71aaac64ef29a0b8e43443c917f308ea4b44e997459ef3630202f9a3db"} Jul 08 18:05:23.267656 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.267632 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hndg9" event={"ID":"bb83970e-f964-4be9-8882-4d1a8b25d3fa","Type":"ContainerStarted","Data":"4a31c221069e50cca1aa39b96442010908d5ca0498e564313a63c09fc755f380"} Jul 08 18:05:23.269060 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.269040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" event={"ID":"3fddd715-cce6-4e4d-a611-adc89869fd3e","Type":"ContainerStarted","Data":"06b56583a47041178ede954e0a0f0fceb4abc56c6a3b875b9f969a5fe631eb9f"} Jul 08 18:05:23.270157 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.270137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" event={"ID":"bf0919cf-dfb8-4f25-8dc2-678163025d26","Type":"ContainerStarted","Data":"dd3a9b99c6691344b361d8d4603a1c4a8326827e705d6da96e4a5501f94b292a"} Jul 08 18:05:23.271292 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.271273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" event={"ID":"d2d495b3-193b-4caf-9329-b9645cebbd40","Type":"ContainerStarted","Data":"24675ad19f8c3de405ed4431cd18557b96ca623d5b07e918dda0ffc192f8707f"} Jul 08 18:05:23.272347 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.272326 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" event={"ID":"07539c45-3389-487c-a42c-b9bb48b09de1","Type":"ContainerStarted","Data":"1b455f42eb13100d2f4b16397f01cf96ff20f10081e014c16674e39cc3869f96"} Jul 08 18:05:23.273797 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.273779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tj2cs" event={"ID":"084a77c4-cf51-49c0-8cb0-1963bec63658","Type":"ContainerStarted","Data":"e801441206506e5464c38af67b4c9b0777197dca5fb116df1818ce602bd958b2"} Jul 08 18:05:23.273904 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.273813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tj2cs" event={"ID":"084a77c4-cf51-49c0-8cb0-1963bec63658","Type":"ContainerStarted","Data":"eae5d4f1140b4066302d19751fa843eb878b4294bbf48a191bce4308d6234874"} Jul 08 18:05:23.274006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.273990 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tj2cs" Jul 08 18:05:23.296729 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.296690 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jq98v" podStartSLOduration=45.696702211 podStartE2EDuration="50.29667777s" totalImagesPullingTime="4.599975559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:23.295570031 +0000 UTC m=+78.980776801" watchObservedRunningTime="2026-07-08 18:05:23.29667777 +0000 UTC m=+78.981884576" Jul 08 18:05:23.332600 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.332545 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-clrnw" podStartSLOduration=70.174193751 podStartE2EDuration="1m15.332522247s" totalImagesPullingTime="5.158328496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:23.329651522 +0000 UTC m=+79.014858293" watchObservedRunningTime="2026-07-08 18:05:23.332522247 +0000 UTC m=+79.017729018" Jul 08 18:05:23.356345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.356306 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:23.432419 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.432361 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tj2cs" podStartSLOduration=33.848839381 podStartE2EDuration="38.432345511s" totalImagesPullingTime="4.58350613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:23.420012105 +0000 UTC m=+79.105218874" watchObservedRunningTime="2026-07-08 18:05:23.432345511 +0000 UTC m=+79.117552494" Jul 08 18:05:23.432648 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.432457 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hndg9" podStartSLOduration=17.200192622 podStartE2EDuration="28.43244976s" totalImagesPullingTime="11.232257138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:23.372758946 +0000 UTC m=+79.057965741" watchObservedRunningTime="2026-07-08 18:05:23.43244976 +0000 UTC m=+79.117656536" Jul 08 18:05:23.487357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.487317 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:23.720934 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.720724 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:23.804613 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:23.804566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:24.125823 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:24.125681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:24.215157 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:24.215123 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:25.621370 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:25.621330 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:25.707283 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:25.707239 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:25.780122 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:25.780093 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:26.469352 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:26.469315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:26.567223 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:26.567124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:26.657155 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:26.656998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:26.769216 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:26.769132 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:26.978330 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:26.978285 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:27.088154 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:27.087768 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:27.763724 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:27.763686 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:27.870933 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:27.870807 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:28.195814 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:28.195781 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:28.297274 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:28.297240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:28.763283 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:28.763252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:28.871261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:28.871119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:28.942417 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:28.942384 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:29.304953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.304917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" event={"ID":"07539c45-3389-487c-a42c-b9bb48b09de1","Type":"ContainerStarted","Data":"66c85495a6b19e8910409eaa339dc058e06d98ee8c0bf3d0239a6c8adb4dc6f9"} Jul 08 18:05:29.305173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.304970 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" event={"ID":"07539c45-3389-487c-a42c-b9bb48b09de1","Type":"ContainerStarted","Data":"dd9cf9b1f7f3f77f254d1a2e5d9ee5bbb7fe58fcee9dc4de4508fd50264a3c04"} Jul 08 18:05:29.305173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.305034 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" event={"ID":"07539c45-3389-487c-a42c-b9bb48b09de1","Type":"ContainerStarted","Data":"f5db5611abeccb769618f1318c516936b9247607191ef519398b4874cee37d14"} Jul 08 18:05:29.306388 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.306358 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" event={"ID":"bf0919cf-dfb8-4f25-8dc2-678163025d26","Type":"ContainerStarted","Data":"adec315db4629c5a86ee3b2e69fb4ac4a4487392edcaeb3a3af042d07cd65203"} Jul 08 18:05:29.307719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.307693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" event={"ID":"d2d495b3-193b-4caf-9329-b9645cebbd40","Type":"ContainerStarted","Data":"a2e6bd708ad002d9ab8cfcf809bb7a0f14f09acd921e4341e40dd9745051461b"} Jul 08 18:05:29.307913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.307896 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:29.309484 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.309465 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" Jul 08 18:05:29.337334 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.337285 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b67d4c6c-bkc5d" podStartSLOduration=4.529035664 podStartE2EDuration="10.337269325s" totalImagesPullingTime="5.808233661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:29.336201236 +0000 UTC m=+85.021408007" watchObservedRunningTime="2026-07-08 18:05:29.337269325 +0000 UTC m=+85.022476094" Jul 08 18:05:29.371606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.371556 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766697ddd7-swf5r" podStartSLOduration=4.585451736 podStartE2EDuration="10.371539281s" totalImagesPullingTime="5.786087545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:29.371083433 +0000 UTC m=+85.056290205" watchObservedRunningTime="2026-07-08 18:05:29.371539281 +0000 UTC m=+85.056746050" Jul 08 18:05:29.402002 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:29.401936 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86b95f8b76-5wjvp" podStartSLOduration=4.611611667 podStartE2EDuration="10.401914655s" totalImagesPullingTime="5.790302988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:29.401130049 +0000 UTC m=+85.086336812" watchObservedRunningTime="2026-07-08 18:05:29.401914655 +0000 UTC m=+85.087121427" Jul 08 18:05:30.863739 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:30.863713 2580 scope.go:117] "RemoveContainer" containerID="ff9e119d4b5a0f1d093cd805a1559b58a43c1d648aad1094e52e0504dc252b4c" Jul 08 18:05:31.317156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:31.317125 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:05:31.317331 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:31.317246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" event={"ID":"6525a595-633b-454a-931a-5c2c55f5b8b4","Type":"ContainerStarted","Data":"bab532263caed905fcc2cb87db8814851ad4d1b4bb1b45dba9433801ab2802da"} Jul 08 18:05:31.317643 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:31.317621 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:05:31.341704 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:31.341649 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" podStartSLOduration=76.353292528 podStartE2EDuration="1m23.341632867s" totalImagesPullingTime="6.988340339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:31.341282307 +0000 UTC m=+87.026489089" watchObservedRunningTime="2026-07-08 18:05:31.341632867 +0000 UTC m=+87.026839635" Jul 08 18:05:31.588644 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:31.588564 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-4vx47" Jul 08 18:05:31.914696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:31.914663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:32.005687 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:32.005648 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:32.099376 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:32.099333 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:32.308446 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:32.308354 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:32.389259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:32.389225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:32.902459 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:32.902425 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:33.163989 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:33.163880 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:33.233810 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:33.233778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:33.281775 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:33.281748 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tj2cs" Jul 08 18:05:33.524975 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:33.524884 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:34.353476 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:34.353444 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:34.418819 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:34.418790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:34.826294 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:34.826260 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:34.892018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:34.891985 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:35.007366 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:35.007321 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:35.648013 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:35.647979 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:35.739923 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:35.739886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:36.671387 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:36.671357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.046604 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.046514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.229903 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.229873 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.360613 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.360515 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.484981 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.484937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.583429 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.583395 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2pl47"] Jul 08 18:05:37.587436 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.587412 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.592063 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.592033 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 18:05:37.592198 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.592037 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 18:05:37.592255 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.592213 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 18:05:37.592331 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.592313 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 18:05:37.592401 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.592386 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xdrn7\"" Jul 08 18:05:37.627225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-wtmp\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-accelerators-collector-config\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-sys\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be2bf90e-0bbf-404d-a675-131be80a9405-metrics-client-ca\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627488 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxf2r\" (UniqueName: \"kubernetes.io/projected/be2bf90e-0bbf-404d-a675-131be80a9405-kube-api-access-kxf2r\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627488 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-root\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627488 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-textfile\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627488 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-tls\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.627488 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.627358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.641609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.641577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.727803 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.727774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-textfile\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.727803 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.727807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-tls\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.727826 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.727885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-wtmp\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.727970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-accelerators-collector-config\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728001 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-wtmp\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-sys\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be2bf90e-0bbf-404d-a675-131be80a9405-metrics-client-ca\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxf2r\" (UniqueName: \"kubernetes.io/projected/be2bf90e-0bbf-404d-a675-131be80a9405-kube-api-access-kxf2r\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-root\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728046 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-sys\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728216 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-textfile\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728226 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be2bf90e-0bbf-404d-a675-131be80a9405-root\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728687 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728631 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-accelerators-collector-config\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.728687 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.728643 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be2bf90e-0bbf-404d-a675-131be80a9405-metrics-client-ca\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.730223 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.730200 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.730355 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.730339 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be2bf90e-0bbf-404d-a675-131be80a9405-node-exporter-tls\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.737598 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.737573 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.743441 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.743418 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxf2r\" (UniqueName: \"kubernetes.io/projected/be2bf90e-0bbf-404d-a675-131be80a9405-kube-api-access-kxf2r\") pod \"node-exporter-2pl47\" (UID: \"be2bf90e-0bbf-404d-a675-131be80a9405\") " pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.747940 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.747911 2580 patch_prober.go:28] interesting pod/image-registry-85fd9b879b-6xmsl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 18:05:37.748035 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.747956 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" podUID="b6457a1b-e229-424b-a634-78951adc295f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 18:05:37.872562 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.872529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:37.897745 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.897722 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2pl47" Jul 08 18:05:37.910725 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:05:37.910694 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2bf90e_0bbf_404d_a675_131be80a9405.slice/crio-040c2c1f167a568b3c5dce915b4231e547867ce4cf131a2276b1c59cd6327730 WatchSource:0}: Error finding container 040c2c1f167a568b3c5dce915b4231e547867ce4cf131a2276b1c59cd6327730: Status 404 returned error can't find the container with id 040c2c1f167a568b3c5dce915b4231e547867ce4cf131a2276b1c59cd6327730 Jul 08 18:05:37.977503 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:37.977457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:38.161858 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:38.161754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:38.345097 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:38.345066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pl47" event={"ID":"be2bf90e-0bbf-404d-a675-131be80a9405","Type":"ContainerStarted","Data":"040c2c1f167a568b3c5dce915b4231e547867ce4cf131a2276b1c59cd6327730"} Jul 08 18:05:38.635992 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:38.635902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:38.859251 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:38.859215 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:39.014021 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:39.013989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:39.246958 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:39.246934 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:05:39.350856 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:39.350751 2580 generic.go:356] "Generic (PLEG): container finished" podID="be2bf90e-0bbf-404d-a675-131be80a9405" containerID="fd3fb6b8a63f331f32d695d563cd0cdb0d7580ded353015a866eedd2d1a5b15c" exitCode=0 Jul 08 18:05:39.350856 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:39.350824 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pl47" event={"ID":"be2bf90e-0bbf-404d-a675-131be80a9405","Type":"ContainerDied","Data":"fd3fb6b8a63f331f32d695d563cd0cdb0d7580ded353015a866eedd2d1a5b15c"} Jul 08 18:05:40.049966 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.049914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:40.154213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.154173 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:40.356971 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.356883 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pl47" event={"ID":"be2bf90e-0bbf-404d-a675-131be80a9405","Type":"ContainerStarted","Data":"7aeaae5a04d316b50192f021d17c77c599575a0249095245f927428441491f19"} Jul 08 18:05:40.356971 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.356939 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2pl47" event={"ID":"be2bf90e-0bbf-404d-a675-131be80a9405","Type":"ContainerStarted","Data":"cbb6c4f16c46b8b395372a4235c9c6e2e3df371e552f9fc1abffff434acbb47f"} Jul 08 18:05:40.390152 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.390104 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2pl47" podStartSLOduration=2.163693727 podStartE2EDuration="3.390091437s" totalImagesPullingTime="1.161639428s" totalInitContainerRuntime="64.758282ms" isStatefulPod=true podCreationTimestamp="2026-07-08 18:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:40.388965481 +0000 UTC m=+96.074172251" watchObservedRunningTime="2026-07-08 18:05:40.390091437 +0000 UTC m=+96.075298234" Jul 08 18:05:40.613431 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.613334 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:40.724494 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.724455 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:40.821360 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:40.821328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:41.076254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:41.076222 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:42.077882 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:42.077847 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:42.246411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:42.246377 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:42.487828 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:42.487784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:42.728827 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:42.728788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:43.201149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:43.201120 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gkkkh" Jul 08 18:05:43.596884 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:43.596784 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85fd9b879b-6xmsl"] Jul 08 18:05:43.723226 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:43.723188 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:43.821532 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:43.821495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:44.102945 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:44.102907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:45.127891 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:45.127859 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:45.201208 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:45.201175 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:45.535059 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:45.535024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:46.608816 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:46.608781 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:46.691101 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:46.691051 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:47.915409 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:47.915381 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:48.035864 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:48.035824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:49.257820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:49.257782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:49.359896 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:49.359857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:49.484521 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:49.484487 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:51.665855 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:51.665813 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:53.764798 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:53.764764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:53.860289 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:53.860262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:58.092511 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:58.092477 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:05:58.191471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:05:58.191433 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:00.144526 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:00.144493 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:00.269292 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:00.269263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:00.588409 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:00.588319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:00.722587 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:00.722542 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:01.792531 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:01.792495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:01.882100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:01.882067 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:02.009529 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:02.009494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:03.549459 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:03.549423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:03.619294 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:03.619257 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:08.620965 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:08.620919 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" podUID="b6457a1b-e229-424b-a634-78951adc295f" containerName="registry" containerID="cri-o://0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4" gracePeriod=30 Jul 08 18:06:08.860720 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:08.860699 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:06:09.007874 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.007828 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-registry-certificates\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.007884 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-image-registry-private-configuration\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.007923 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-trusted-ca\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.007949 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6457a1b-e229-424b-a634-78951adc295f-ca-trust-extracted\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.007980 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-bound-sa-token\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.008004 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5vkf\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-kube-api-access-z5vkf\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.008042 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.008077 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-installation-pull-secrets\") pod \"b6457a1b-e229-424b-a634-78951adc295f\" (UID: \"b6457a1b-e229-424b-a634-78951adc295f\") " Jul 08 18:06:09.008471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.008396 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:06:09.008471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.008425 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:06:09.010916 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.010863 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:06:09.011168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.011120 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:06:09.011263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.011221 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-kube-api-access-z5vkf" (OuterVolumeSpecName: "kube-api-access-z5vkf") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "kube-api-access-z5vkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:06:09.011263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.011244 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:06:09.011380 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.011358 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:06:09.018662 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.018641 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6457a1b-e229-424b-a634-78951adc295f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b6457a1b-e229-424b-a634-78951adc295f" (UID: "b6457a1b-e229-424b-a634-78951adc295f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:06:09.109371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109338 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-registry-certificates\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.109371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109367 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-image-registry-private-configuration\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.109537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109381 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6457a1b-e229-424b-a634-78951adc295f-trusted-ca\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.109537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109394 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6457a1b-e229-424b-a634-78951adc295f-ca-trust-extracted\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.109537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109406 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-bound-sa-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.109537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109417 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5vkf\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-kube-api-access-z5vkf\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.109537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109430 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6457a1b-e229-424b-a634-78951adc295f-registry-tls\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.109537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.109442 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6457a1b-e229-424b-a634-78951adc295f-installation-pull-secrets\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:06:09.467569 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.467538 2580 generic.go:356] "Generic (PLEG): container finished" podID="a5195670-b3c0-4736-a5dc-3faab67e0195" containerID="83c3171981723898e95dc222def9fa7b144c30de0181919d0eda547317ba3bc3" exitCode=0 Jul 08 18:06:09.467718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.467610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" event={"ID":"a5195670-b3c0-4736-a5dc-3faab67e0195","Type":"ContainerDied","Data":"83c3171981723898e95dc222def9fa7b144c30de0181919d0eda547317ba3bc3"} Jul 08 18:06:09.467997 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.467976 2580 scope.go:117] "RemoveContainer" containerID="83c3171981723898e95dc222def9fa7b144c30de0181919d0eda547317ba3bc3" Jul 08 18:06:09.468850 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.468812 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6457a1b-e229-424b-a634-78951adc295f" containerID="0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4" exitCode=0 Jul 08 18:06:09.468968 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.468884 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" Jul 08 18:06:09.468968 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.468909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" event={"ID":"b6457a1b-e229-424b-a634-78951adc295f","Type":"ContainerDied","Data":"0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4"} Jul 08 18:06:09.468968 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.468952 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd9b879b-6xmsl" event={"ID":"b6457a1b-e229-424b-a634-78951adc295f","Type":"ContainerDied","Data":"7b28edb61e0c83c9d552cdefff2c129ef309f2fdb1eee71c3d3c867ac803b468"} Jul 08 18:06:09.469129 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.469012 2580 scope.go:117] "RemoveContainer" containerID="0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4" Jul 08 18:06:09.470568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.470553 2580 generic.go:356] "Generic (PLEG): container finished" podID="4f7d7b3b-c770-4330-ba51-218d3b049f6b" containerID="fe7644d8c20a1f3d63f060adb31742e242653a631f8d2ebb5319f551336fe85c" exitCode=0 Jul 08 18:06:09.470641 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.470626 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" event={"ID":"4f7d7b3b-c770-4330-ba51-218d3b049f6b","Type":"ContainerDied","Data":"fe7644d8c20a1f3d63f060adb31742e242653a631f8d2ebb5319f551336fe85c"} Jul 08 18:06:09.471036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.471017 2580 scope.go:117] "RemoveContainer" containerID="fe7644d8c20a1f3d63f060adb31742e242653a631f8d2ebb5319f551336fe85c" Jul 08 18:06:09.480226 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.480206 2580 scope.go:117] "RemoveContainer" containerID="0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4" Jul 08 18:06:09.480570 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:06:09.480523 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4\": container with ID starting with 0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4 not found: ID does not exist" containerID="0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4" Jul 08 18:06:09.480637 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.480558 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4"} err="failed to get container status \"0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4\": rpc error: code = NotFound desc = could not find container \"0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4\": container with ID starting with 0d4eace88ab22020cc6a5601949c49a9df43a94f8132d1108f25806fdd4babb4 not found: ID does not exist" Jul 08 18:06:09.555642 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.555571 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85fd9b879b-6xmsl"] Jul 08 18:06:09.561895 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.561867 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85fd9b879b-6xmsl"] Jul 08 18:06:09.575387 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.575347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:09.710670 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:09.710644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:10.252133 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:10.252099 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d459b6969-8sg97_6c74951d-5706-4a46-9163-5b9d3dc67042/router/0.log" Jul 08 18:06:10.289640 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:10.289613 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xsrcw_3c398807-d5ee-4ab3-9cf6-94364606b1d6/serve-healthcheck-canary/0.log" Jul 08 18:06:10.476213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:10.476176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6rsgg" event={"ID":"a5195670-b3c0-4736-a5dc-3faab67e0195","Type":"ContainerStarted","Data":"24e03dbef7fd9a20df5983d70da732570a708aa73abb903f258740bec2fda599"} Jul 08 18:06:10.478822 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:10.478797 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-lnv6h" event={"ID":"4f7d7b3b-c770-4330-ba51-218d3b049f6b","Type":"ContainerStarted","Data":"45352e47da56e257963b5eb20a039da7d52af4a69ba445486e31faed94e5d9bb"} Jul 08 18:06:10.869161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:10.869118 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6457a1b-e229-424b-a634-78951adc295f" path="/var/lib/kubelet/pods/b6457a1b-e229-424b-a634-78951adc295f/volumes" Jul 08 18:06:15.178048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:15.178013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:15.271864 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:15.271797 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:17.053280 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:17.053248 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:20.518238 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:20.518210 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c57f641-7b79-42b9-a796-35eb35a22fc6" containerID="9e3fdeef84a37c75a0cbe4edfbda645356da0fab9077a6dfc43fc2a1e5fcb8ec" exitCode=0 Jul 08 18:06:20.518616 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:20.518289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" event={"ID":"7c57f641-7b79-42b9-a796-35eb35a22fc6","Type":"ContainerDied","Data":"9e3fdeef84a37c75a0cbe4edfbda645356da0fab9077a6dfc43fc2a1e5fcb8ec"} Jul 08 18:06:20.518789 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:20.518770 2580 scope.go:117] "RemoveContainer" containerID="9e3fdeef84a37c75a0cbe4edfbda645356da0fab9077a6dfc43fc2a1e5fcb8ec" Jul 08 18:06:21.525900 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:21.525821 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wf66x" event={"ID":"7c57f641-7b79-42b9-a796-35eb35a22fc6","Type":"ContainerStarted","Data":"64ebae00a808335dbb4de952adc9a14615cae2ee4360b552cf4ea0875c0491a1"} Jul 08 18:06:21.747334 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:21.747299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:21.903515 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:21.903471 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:06:21.997225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:06:21.997169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:04.021333 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:04.021245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:04.124846 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:04.124808 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:15.936814 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:15.936779 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:16.014930 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:16.014899 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:30.131722 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:30.131685 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:30.231912 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:30.231876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:30.333893 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:30.333857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:31.950944 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:31.950915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:32.067156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:32.067124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:39.840616 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:39.840579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:40.288482 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:40.288446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:40.387189 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:40.387117 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:43.272629 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:43.272595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:43.373546 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:43.373505 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:07:43.472591 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:07:43.472555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:08:21.418560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:08:21.418524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:08:36.287144 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:08:36.287051 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:08:36.368772 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:08:36.368743 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:04.716572 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:04.716544 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:09:04.724678 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:04.724651 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:09:04.728172 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:04.728151 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 18:09:07.238650 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:07.238616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:09.571135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:09.571100 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:12.392809 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:12.392775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:13.445496 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:13.445462 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:13.945915 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:13.945882 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:14.087525 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:14.087499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:14.607010 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:14.606935 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:14.720913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:14.720884 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:16.386991 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:16.386942 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:16.984258 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:16.984217 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:19.013483 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:19.013446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:19.115602 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:19.115560 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:23.741883 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:23.741821 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:23.851171 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:23.851140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:25.057634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:25.057597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:25.820676 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:25.820641 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:27.578161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:27.578129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:27.680233 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:27.680199 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:33.299804 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:33.299774 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:33.392684 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:33.392644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:35.548050 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:35.548021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:50.995701 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:50.995663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:51.104481 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:51.104453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:52.368694 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.368660 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh"] Jul 08 18:09:52.369106 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.368993 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6457a1b-e229-424b-a634-78951adc295f" containerName="registry" Jul 08 18:09:52.369106 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.369004 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6457a1b-e229-424b-a634-78951adc295f" containerName="registry" Jul 08 18:09:52.369106 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.369057 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6457a1b-e229-424b-a634-78951adc295f" containerName="registry" Jul 08 18:09:52.372066 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.372046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.375140 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.375124 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 18:09:52.375257 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.375125 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 18:09:52.375414 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.375402 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 18:09:52.377799 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.377782 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 18:09:52.385873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.385855 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pxw5w\"" Jul 08 18:09:52.402646 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.402623 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh"] Jul 08 18:09:52.493903 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.493876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:52.534904 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.534874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.535272 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.535215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bqbn\" (UniqueName: \"kubernetes.io/projected/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-kube-api-access-8bqbn\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.535504 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.535488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-manifests\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.535725 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.535710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-tmp\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.535929 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.535914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.612008 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.611973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:52.636992 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.636973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.637100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.637002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bqbn\" (UniqueName: \"kubernetes.io/projected/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-kube-api-access-8bqbn\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.637100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.637036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-manifests\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.637100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.637062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-tmp\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.637100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.637080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.637401 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.637383 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-manifests\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.639335 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.639314 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-tmp\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.639572 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.639556 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.639611 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.639599 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.666577 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.666554 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bqbn\" (UniqueName: \"kubernetes.io/projected/8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f-kube-api-access-8bqbn\") pod \"opendatahub-operator-controller-manager-859965ff4-z59wh\" (UID: \"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.681404 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.681386 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:09:52.713821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.713788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:52.822995 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.822968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:52.832105 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.832077 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh"] Jul 08 18:09:52.836119 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:09:52.836095 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbc1eaa_8eaa_4950_81e4_72aee32a7f7f.slice/crio-f39f5a86f6c2129ca2b054f3b90376b11d7460d760f467f4ccf52a64ff1ad493 WatchSource:0}: Error finding container f39f5a86f6c2129ca2b054f3b90376b11d7460d760f467f4ccf52a64ff1ad493: Status 404 returned error can't find the container with id f39f5a86f6c2129ca2b054f3b90376b11d7460d760f467f4ccf52a64ff1ad493 Jul 08 18:09:52.837784 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.837769 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:09:52.912687 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:52.912618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:53.044505 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:53.044476 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:53.174140 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:53.174059 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:53.263560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:53.263530 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:53.320344 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:53.320315 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" event={"ID":"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f","Type":"ContainerStarted","Data":"f39f5a86f6c2129ca2b054f3b90376b11d7460d760f467f4ccf52a64ff1ad493"} Jul 08 18:09:53.400255 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:53.400229 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:09:58.343317 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:58.343275 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" event={"ID":"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f","Type":"ContainerStarted","Data":"ea329c20a3ef7864a3456702fb7f2d5438ccbc5edc01f578a5cb4c2fe77ee536"} Jul 08 18:09:59.349220 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:59.349183 2580 generic.go:356] "Generic (PLEG): container finished" podID="8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f" containerID="ea329c20a3ef7864a3456702fb7f2d5438ccbc5edc01f578a5cb4c2fe77ee536" exitCode=0 Jul 08 18:09:59.349599 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:09:59.349227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" event={"ID":"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f","Type":"ContainerDied","Data":"ea329c20a3ef7864a3456702fb7f2d5438ccbc5edc01f578a5cb4c2fe77ee536"} Jul 08 18:10:00.355420 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:00.355387 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" event={"ID":"8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f","Type":"ContainerStarted","Data":"c823b241a02f16033176b692753fc9ea542693c1f746c7c57aff7cab759e16b0"} Jul 08 18:10:00.355805 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:00.355524 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:10:00.382326 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:00.382268 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" podStartSLOduration=2.273185066 podStartE2EDuration="8.382253756s" totalImagesPullingTime="5.053696554s" totalInitContainerRuntime="1.055372136s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:09:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:00.379914709 +0000 UTC m=+356.065121474" watchObservedRunningTime="2026-07-08 18:10:00.382253756 +0000 UTC m=+356.067460525" Jul 08 18:10:11.361339 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:11.361307 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-z59wh" Jul 08 18:10:13.315082 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:13.315017 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:13.519926 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:13.519886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:14.129134 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.129098 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:14.193810 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.193782 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6784757fb-29tmb"] Jul 08 18:10:14.206138 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.206117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.209212 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.209183 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 18:10:14.209212 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.209194 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 18:10:14.214534 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.214510 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6784757fb-29tmb"] Jul 08 18:10:14.307338 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.307310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbn8d\" (UniqueName: \"kubernetes.io/projected/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-kube-api-access-hbn8d\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.307491 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.307360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-tmp\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.307491 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.307449 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-tls-certs\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.312185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.312162 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-smjqr"] Jul 08 18:10:14.315393 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.315378 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:14.317695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.317679 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5nzn2\"" Jul 08 18:10:14.317865 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.317848 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 18:10:14.342261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.342234 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-smjqr"] Jul 08 18:10:14.349241 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.349215 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:14.408528 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.408497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbn8d\" (UniqueName: \"kubernetes.io/projected/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-kube-api-access-hbn8d\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.408674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.408543 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxqkl\" (UniqueName: \"kubernetes.io/projected/00222fa8-23cb-4a82-bdfb-154cb3663b3e-kube-api-access-hxqkl\") pod \"odh-model-controller-7998c67dc4-smjqr\" (UID: \"00222fa8-23cb-4a82-bdfb-154cb3663b3e\") " pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:14.408674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.408603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-tmp\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.408674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.408650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00222fa8-23cb-4a82-bdfb-154cb3663b3e-cert\") pod \"odh-model-controller-7998c67dc4-smjqr\" (UID: \"00222fa8-23cb-4a82-bdfb-154cb3663b3e\") " pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:14.408829 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.408679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-tls-certs\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.410938 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.410908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-tmp\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.411287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.411268 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-tls-certs\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.420486 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.420462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbn8d\" (UniqueName: \"kubernetes.io/projected/5ba67f36-15a0-4cd4-9dd5-2376c770a2c9-kube-api-access-hbn8d\") pod \"kube-auth-proxy-6784757fb-29tmb\" (UID: \"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.509963 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.509930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00222fa8-23cb-4a82-bdfb-154cb3663b3e-cert\") pod \"odh-model-controller-7998c67dc4-smjqr\" (UID: \"00222fa8-23cb-4a82-bdfb-154cb3663b3e\") " pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:14.510118 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.509980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxqkl\" (UniqueName: \"kubernetes.io/projected/00222fa8-23cb-4a82-bdfb-154cb3663b3e-kube-api-access-hxqkl\") pod \"odh-model-controller-7998c67dc4-smjqr\" (UID: \"00222fa8-23cb-4a82-bdfb-154cb3663b3e\") " pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:14.510159 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:10:14.510109 2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 18:10:14.510200 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:10:14.510182 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00222fa8-23cb-4a82-bdfb-154cb3663b3e-cert podName:00222fa8-23cb-4a82-bdfb-154cb3663b3e nodeName:}" failed. No retries permitted until 2026-07-08 18:10:15.010162245 +0000 UTC m=+370.695368997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/00222fa8-23cb-4a82-bdfb-154cb3663b3e-cert") pod "odh-model-controller-7998c67dc4-smjqr" (UID: "00222fa8-23cb-4a82-bdfb-154cb3663b3e") : secret "odh-model-controller-webhook-cert" not found Jul 08 18:10:14.516298 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.516272 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" Jul 08 18:10:14.521004 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.520970 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxqkl\" (UniqueName: \"kubernetes.io/projected/00222fa8-23cb-4a82-bdfb-154cb3663b3e-kube-api-access-hxqkl\") pod \"odh-model-controller-7998c67dc4-smjqr\" (UID: \"00222fa8-23cb-4a82-bdfb-154cb3663b3e\") " pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:14.522403 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.522374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:14.677498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.677423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:14.708283 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.708248 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6784757fb-29tmb"] Jul 08 18:10:14.712177 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:10:14.712154 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba67f36_15a0_4cd4_9dd5_2376c770a2c9.slice/crio-5603d2f6ce3fb2a04d19dd55be54568cc4f79cbd882d644b1127443d64cc765b WatchSource:0}: Error finding container 5603d2f6ce3fb2a04d19dd55be54568cc4f79cbd882d644b1127443d64cc765b: Status 404 returned error can't find the container with id 5603d2f6ce3fb2a04d19dd55be54568cc4f79cbd882d644b1127443d64cc765b Jul 08 18:10:14.962498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:14.962417 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:15.014470 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.014435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00222fa8-23cb-4a82-bdfb-154cb3663b3e-cert\") pod \"odh-model-controller-7998c67dc4-smjqr\" (UID: \"00222fa8-23cb-4a82-bdfb-154cb3663b3e\") " pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:15.016892 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.016868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00222fa8-23cb-4a82-bdfb-154cb3663b3e-cert\") pod \"odh-model-controller-7998c67dc4-smjqr\" (UID: \"00222fa8-23cb-4a82-bdfb-154cb3663b3e\") " pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:15.224968 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.224884 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:15.272416 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.272384 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r662d"] Jul 08 18:10:15.276601 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.276087 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:15.289936 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.289908 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 18:10:15.310008 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.309977 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r662d"] Jul 08 18:10:15.318280 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.318247 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mchjm\" (UniqueName: \"kubernetes.io/projected/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-kube-api-access-mchjm\") pod \"model-serving-api-54d89bd5fc-r662d\" (UID: \"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:15.318673 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.318318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-r662d\" (UID: \"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:15.339590 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.339551 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:15.385568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.385543 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-smjqr"] Jul 08 18:10:15.387861 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:10:15.387813 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00222fa8_23cb_4a82_bdfb_154cb3663b3e.slice/crio-e4c1443eeba56241dd7045542f3fe5e50e4dacd18468fba23ebb6106a2a8fdf7 WatchSource:0}: Error finding container e4c1443eeba56241dd7045542f3fe5e50e4dacd18468fba23ebb6106a2a8fdf7: Status 404 returned error can't find the container with id e4c1443eeba56241dd7045542f3fe5e50e4dacd18468fba23ebb6106a2a8fdf7 Jul 08 18:10:15.418470 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.418437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" event={"ID":"00222fa8-23cb-4a82-bdfb-154cb3663b3e","Type":"ContainerStarted","Data":"e4c1443eeba56241dd7045542f3fe5e50e4dacd18468fba23ebb6106a2a8fdf7"} Jul 08 18:10:15.418730 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.418702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-r662d\" (UID: \"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:15.418869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.418792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mchjm\" (UniqueName: \"kubernetes.io/projected/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-kube-api-access-mchjm\") pod \"model-serving-api-54d89bd5fc-r662d\" (UID: \"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:15.418869 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:10:15.418814 2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 08 18:10:15.418987 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:10:15.418901 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-tls-certs podName:67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5 nodeName:}" failed. No retries permitted until 2026-07-08 18:10:15.918881067 +0000 UTC m=+371.604087872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-tls-certs") pod "model-serving-api-54d89bd5fc-r662d" (UID: "67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5") : secret "model-serving-api-tls" not found Jul 08 18:10:15.423848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.423804 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" event={"ID":"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9","Type":"ContainerStarted","Data":"5603d2f6ce3fb2a04d19dd55be54568cc4f79cbd882d644b1127443d64cc765b"} Jul 08 18:10:15.489394 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.489328 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mchjm\" (UniqueName: \"kubernetes.io/projected/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-kube-api-access-mchjm\") pod \"model-serving-api-54d89bd5fc-r662d\" (UID: \"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:15.660802 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.660761 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:15.922706 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.922670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-r662d\" (UID: \"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:15.925130 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:15.925105 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-r662d\" (UID: \"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:16.169148 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.169107 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll"] Jul 08 18:10:16.171691 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.171647 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:16.173888 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.173846 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.176366 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.176347 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 18:10:16.176475 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.176401 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 18:10:16.176475 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.176421 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gxcj2\"" Jul 08 18:10:16.176607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.176493 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 18:10:16.176721 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.176706 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 18:10:16.190081 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.190052 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:16.192409 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.192378 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll"] Jul 08 18:10:16.226387 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.226353 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-cacerts\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.226387 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.226388 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-token\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.226582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.226417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.226582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.226475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.226582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.226498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.226582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.226559 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-local-certs\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.226727 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.226594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw25t\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-kube-api-access-rw25t\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.260046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.260011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:16.327632 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.327600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.328135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.327649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.328135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.327680 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.328135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.327711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-local-certs\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.328135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.327762 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw25t\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-kube-api-access-rw25t\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.328135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.327807 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-cacerts\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.328135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.327882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-token\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.328478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.328448 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.330861 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.330790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-local-certs\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.330991 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.330895 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.331167 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.331147 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.331232 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.331179 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-cacerts\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.378201 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.378172 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-token\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.385115 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.385088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw25t\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-kube-api-access-rw25t\") pod \"istiod-openshift-gateway-7574459656-vj8ll\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.404494 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.404474 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r662d"] Jul 08 18:10:16.406232 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:10:16.406196 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e3b6a2_a0bb_4d15_a585_5b6fac75c7e5.slice/crio-f5ba5e0155a79fb41c38a2f6873a3f4c1061e8e0d86269d4f6e38fe4913d7bdd WatchSource:0}: Error finding container f5ba5e0155a79fb41c38a2f6873a3f4c1061e8e0d86269d4f6e38fe4913d7bdd: Status 404 returned error can't find the container with id f5ba5e0155a79fb41c38a2f6873a3f4c1061e8e0d86269d4f6e38fe4913d7bdd Jul 08 18:10:16.429202 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.429134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r662d" event={"ID":"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5","Type":"ContainerStarted","Data":"f5ba5e0155a79fb41c38a2f6873a3f4c1061e8e0d86269d4f6e38fe4913d7bdd"} Jul 08 18:10:16.483712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.483675 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:16.641460 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:16.641430 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll"] Jul 08 18:10:16.642410 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:10:16.642386 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1cb37c_1f29_47a2_8aa5_a2ca30dda4e1.slice/crio-00dfa13c26976a1b29d8ab334d7a8cec41ed76da5fe8e164e5edeb133ef3d752 WatchSource:0}: Error finding container 00dfa13c26976a1b29d8ab334d7a8cec41ed76da5fe8e164e5edeb133ef3d752: Status 404 returned error can't find the container with id 00dfa13c26976a1b29d8ab334d7a8cec41ed76da5fe8e164e5edeb133ef3d752 Jul 08 18:10:17.434189 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:17.434159 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" event={"ID":"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1","Type":"ContainerStarted","Data":"00dfa13c26976a1b29d8ab334d7a8cec41ed76da5fe8e164e5edeb133ef3d752"} Jul 08 18:10:17.627208 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:17.627163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:21.730209 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:21.730163 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:10:21.730500 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:21.730263 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:10:22.463309 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.463271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" event={"ID":"00222fa8-23cb-4a82-bdfb-154cb3663b3e","Type":"ContainerStarted","Data":"6bce3e21b0ff39cfe2c58dde7702bf67dd603e19c26ce3baf0e975721d8c9ef2"} Jul 08 18:10:22.463509 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.463397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:22.465026 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.464996 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r662d" event={"ID":"67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5","Type":"ContainerStarted","Data":"a485f35fd9ffa59191f58816234c4dc176af9e6e16109337f840e125e675ee9c"} Jul 08 18:10:22.465217 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.465049 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:22.466517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.466488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" event={"ID":"5ba67f36-15a0-4cd4-9dd5-2376c770a2c9","Type":"ContainerStarted","Data":"940bb61479db410d03ee5022548d43b3c98b762088a880a8e2ce6ff97461a5f4"} Jul 08 18:10:22.468012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.467994 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" event={"ID":"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1","Type":"ContainerStarted","Data":"c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58"} Jul 08 18:10:22.468199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.468182 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:22.469989 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.469970 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vj8ll container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 18:10:22.470067 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.470011 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" podUID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 18:10:22.497396 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.497347 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" podStartSLOduration=2.146050298 podStartE2EDuration="8.497332243s" totalImagesPullingTime="6.351281945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:22.497270125 +0000 UTC m=+378.182476904" watchObservedRunningTime="2026-07-08 18:10:22.497332243 +0000 UTC m=+378.182539009" Jul 08 18:10:22.532618 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.532565 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6784757fb-29tmb" podStartSLOduration=1.506106352 podStartE2EDuration="8.53254907s" totalImagesPullingTime="7.026442718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:22.532235291 +0000 UTC m=+378.217442063" watchObservedRunningTime="2026-07-08 18:10:22.53254907 +0000 UTC m=+378.217755839" Jul 08 18:10:22.606850 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:22.606787 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-r662d" podStartSLOduration=2.285725652 podStartE2EDuration="7.606770243s" totalImagesPullingTime="5.321044591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:22.569614139 +0000 UTC m=+378.254820908" watchObservedRunningTime="2026-07-08 18:10:22.606770243 +0000 UTC m=+378.291977042" Jul 08 18:10:23.377568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.377530 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:23.473453 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.473420 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vj8ll container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 18:10:23.473633 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.473476 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" podUID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 18:10:23.483236 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.483202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:23.732214 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.732153 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" podStartSLOduration=2.646657334 podStartE2EDuration="7.732129641s" totalImagesPullingTime="5.085472307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:22.606476505 +0000 UTC m=+378.291683275" watchObservedRunningTime="2026-07-08 18:10:23.732129641 +0000 UTC m=+379.417336411" Jul 08 18:10:23.733309 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.733266 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz"] Jul 08 18:10:23.737325 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.737304 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.741537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.741521 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xz82t\"" Jul 08 18:10:23.745228 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.745204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:23.762639 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.762609 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz"] Jul 08 18:10:23.794466 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794587 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794587 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794587 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794596 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794656 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9glf\" (UniqueName: \"kubernetes.io/projected/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-kube-api-access-v9glf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.794712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.794708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.895820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.895788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.895820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.895829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896076 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.895896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896076 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.895919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896076 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.895944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896076 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.895980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896076 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896076 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896060 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896361 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896110 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9glf\" (UniqueName: \"kubernetes.io/projected/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-kube-api-access-v9glf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896419 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896371 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896419 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896393 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896486 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896497 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.896698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.896680 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.898381 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.898355 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.898702 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.898679 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.910978 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.910945 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9glf\" (UniqueName: \"kubernetes.io/projected/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-kube-api-access-v9glf\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.914664 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.914642 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a37a64-1b14-4e6a-a15c-91ed60294d7f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wscxz\" (UID: \"c6a37a64-1b14-4e6a-a15c-91ed60294d7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:23.972972 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:23.972936 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:24.048885 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:24.048791 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:24.086036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:24.086001 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:24.204416 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:24.204395 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz"] Jul 08 18:10:24.206584 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:10:24.206559 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a37a64_1b14_4e6a_a15c_91ed60294d7f.slice/crio-add1294db4e18a17a3d9fbe81b4514ebd677d1fc28da142b039577b334b9ff21 WatchSource:0}: Error finding container add1294db4e18a17a3d9fbe81b4514ebd677d1fc28da142b039577b334b9ff21: Status 404 returned error can't find the container with id add1294db4e18a17a3d9fbe81b4514ebd677d1fc28da142b039577b334b9ff21 Jul 08 18:10:24.478454 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:24.478418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" event={"ID":"c6a37a64-1b14-4e6a-a15c-91ed60294d7f","Type":"ContainerStarted","Data":"add1294db4e18a17a3d9fbe81b4514ebd677d1fc28da142b039577b334b9ff21"} Jul 08 18:10:26.424895 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.424846 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:26.474678 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.474435 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:26.543515 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.543475 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:26.582513 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.582482 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm"] Jul 08 18:10:26.589850 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.589806 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.599423 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.599398 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 18:10:26.599423 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.599411 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qg2gc\"" Jul 08 18:10:26.617531 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.617502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm"] Jul 08 18:10:26.665695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.665665 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:26.719796 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.719715 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fddb3ef-91b0-4934-b9c5-838e4324b48e-cert\") pod \"kserve-controller-manager-d6f4b65b8-r6vjm\" (UID: \"2fddb3ef-91b0-4934-b9c5-838e4324b48e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.719796 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.719788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-992fg\" (UniqueName: \"kubernetes.io/projected/2fddb3ef-91b0-4934-b9c5-838e4324b48e-kube-api-access-992fg\") pod \"kserve-controller-manager-d6f4b65b8-r6vjm\" (UID: \"2fddb3ef-91b0-4934-b9c5-838e4324b48e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.821223 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.821189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-992fg\" (UniqueName: \"kubernetes.io/projected/2fddb3ef-91b0-4934-b9c5-838e4324b48e-kube-api-access-992fg\") pod \"kserve-controller-manager-d6f4b65b8-r6vjm\" (UID: \"2fddb3ef-91b0-4934-b9c5-838e4324b48e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.821411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.821279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fddb3ef-91b0-4934-b9c5-838e4324b48e-cert\") pod \"kserve-controller-manager-d6f4b65b8-r6vjm\" (UID: \"2fddb3ef-91b0-4934-b9c5-838e4324b48e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.824085 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.824057 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2fddb3ef-91b0-4934-b9c5-838e4324b48e-cert\") pod \"kserve-controller-manager-d6f4b65b8-r6vjm\" (UID: \"2fddb3ef-91b0-4934-b9c5-838e4324b48e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.834533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.834505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-992fg\" (UniqueName: \"kubernetes.io/projected/2fddb3ef-91b0-4934-b9c5-838e4324b48e-kube-api-access-992fg\") pod \"kserve-controller-manager-d6f4b65b8-r6vjm\" (UID: \"2fddb3ef-91b0-4934-b9c5-838e4324b48e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.858869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.858798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:26.903974 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.903941 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:26.958309 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:26.958276 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:27.175982 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.175938 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:27.229003 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.228942 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:10:27.229115 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.229024 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:10:27.229115 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.229059 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:10:27.322908 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.322882 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm"] Jul 08 18:10:27.325424 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:10:27.325399 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fddb3ef_91b0_4934_b9c5_838e4324b48e.slice/crio-c5ded2ef87b49c6e27e3692b2a9a99fcef07386203c3f5c35cb4b827701687ba WatchSource:0}: Error finding container c5ded2ef87b49c6e27e3692b2a9a99fcef07386203c3f5c35cb4b827701687ba: Status 404 returned error can't find the container with id c5ded2ef87b49c6e27e3692b2a9a99fcef07386203c3f5c35cb4b827701687ba Jul 08 18:10:27.494612 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.494533 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" event={"ID":"2fddb3ef-91b0-4934-b9c5-838e4324b48e","Type":"ContainerStarted","Data":"c5ded2ef87b49c6e27e3692b2a9a99fcef07386203c3f5c35cb4b827701687ba"} Jul 08 18:10:27.496270 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.496240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" event={"ID":"c6a37a64-1b14-4e6a-a15c-91ed60294d7f","Type":"ContainerStarted","Data":"ace3d6792f03b1bacf311a7f7aa145d11ad747c08797e300eec031e924be71f5"} Jul 08 18:10:27.528620 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:27.528566 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" podStartSLOduration=1.508402297 podStartE2EDuration="4.528548693s" totalImagesPullingTime="3.020146396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:27.526926581 +0000 UTC m=+383.212133350" watchObservedRunningTime="2026-07-08 18:10:27.528548693 +0000 UTC m=+383.213755458" Jul 08 18:10:28.049676 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:28.049642 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:28.054432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:28.054407 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:28.500305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:28.500273 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:28.501287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:28.501268 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wscxz" Jul 08 18:10:29.921921 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:29.921888 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll"] Jul 08 18:10:29.922288 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:29.922075 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" podUID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" containerName="discovery" containerID="cri-o://c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58" gracePeriod=30 Jul 08 18:10:30.173863 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.173784 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:30.257534 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.257493 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw25t\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-kube-api-access-rw25t\") pod \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " Jul 08 18:10:30.257709 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.257559 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-token\") pod \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " Jul 08 18:10:30.257709 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.257592 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-local-certs\") pod \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " Jul 08 18:10:30.257709 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.257620 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-kubeconfig\") pod \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " Jul 08 18:10:30.257955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.257767 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-dns-cert\") pod \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " Jul 08 18:10:30.257955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.257865 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-cacerts\") pod \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " Jul 08 18:10:30.257955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.257917 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-ca-configmap\") pod \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\" (UID: \"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1\") " Jul 08 18:10:30.258885 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.258851 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" (UID: "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:10:30.260643 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.260602 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-cacerts" (OuterVolumeSpecName: "cacerts") pod "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" (UID: "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:10:30.260998 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.260963 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" (UID: "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:10:30.261091 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.261001 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-kube-api-access-rw25t" (OuterVolumeSpecName: "kube-api-access-rw25t") pod "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" (UID: "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1"). InnerVolumeSpecName "kube-api-access-rw25t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:10:30.262495 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.262472 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" (UID: "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:10:30.263750 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.263721 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-local-certs" (OuterVolumeSpecName: "local-certs") pod "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" (UID: "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:10:30.264136 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.264112 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" (UID: "9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:10:30.359367 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.359337 2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-cacerts\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:10:30.359367 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.359363 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-ca-configmap\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:10:30.359367 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.359374 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw25t\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-kube-api-access-rw25t\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:10:30.359578 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.359383 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:10:30.359578 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.359392 2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-local-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:10:30.359578 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.359400 2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-kubeconfig\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:10:30.359578 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.359407 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1-istio-csr-dns-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:10:30.509696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.509610 2580 generic.go:356] "Generic (PLEG): container finished" podID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" containerID="c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58" exitCode=0 Jul 08 18:10:30.509696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.509671 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" Jul 08 18:10:30.509944 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.509695 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" event={"ID":"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1","Type":"ContainerDied","Data":"c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58"} Jul 08 18:10:30.509944 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.509746 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll" event={"ID":"9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1","Type":"ContainerDied","Data":"00dfa13c26976a1b29d8ab334d7a8cec41ed76da5fe8e164e5edeb133ef3d752"} Jul 08 18:10:30.509944 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.509765 2580 scope.go:117] "RemoveContainer" containerID="c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58" Jul 08 18:10:30.519186 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.519172 2580 scope.go:117] "RemoveContainer" containerID="c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58" Jul 08 18:10:30.519434 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:10:30.519417 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58\": container with ID starting with c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58 not found: ID does not exist" containerID="c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58" Jul 08 18:10:30.519505 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.519445 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58"} err="failed to get container status \"c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58\": rpc error: code = NotFound desc = could not find container \"c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58\": container with ID starting with c5fd64cfac7a0aff7daea821bf22d4a4490b712a276c9a4541864a2e91bcff58 not found: ID does not exist" Jul 08 18:10:30.536737 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.536708 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll"] Jul 08 18:10:30.539900 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.539880 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vj8ll"] Jul 08 18:10:30.871462 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:30.871388 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" path="/var/lib/kubelet/pods/9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1/volumes" Jul 08 18:10:32.521110 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:32.521070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" event={"ID":"2fddb3ef-91b0-4934-b9c5-838e4324b48e","Type":"ContainerStarted","Data":"b7af286ac994dd854775de56435a1aada9053da8a6ca97170bde034c7a8188d6"} Jul 08 18:10:32.521630 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:32.521163 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:10:33.474897 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:33.474868 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-smjqr" Jul 08 18:10:33.477005 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:33.476977 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-r662d" Jul 08 18:10:33.519077 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:33.519029 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" podStartSLOduration=3.318006637 podStartE2EDuration="7.519013311s" totalImagesPullingTime="4.201006674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:32.556913905 +0000 UTC m=+388.242120675" watchObservedRunningTime="2026-07-08 18:10:33.519013311 +0000 UTC m=+389.204220080" Jul 08 18:10:51.491961 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:51.491923 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:51.600136 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:51.600099 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:53.475534 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:53.475498 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:53.578297 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:53.578265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:55.008968 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:55.008936 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:55.089799 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:55.089767 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:55.234683 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:55.234649 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:55.330894 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:55.330806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:55.869107 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:55.869073 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:10:56.010913 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:10:56.010877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:03.531332 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:03.531302 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r6vjm" Jul 08 18:11:06.256345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:06.256299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:06.360655 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:06.360623 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:14.925671 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:14.925643 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:15.017996 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.017963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:15.054086 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.054054 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hf5q"] Jul 08 18:11:15.054383 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.054372 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" containerName="discovery" Jul 08 18:11:15.054424 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.054385 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" containerName="discovery" Jul 08 18:11:15.054458 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.054445 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1cb37c-1f29-47a2-8aa5-a2ca30dda4e1" containerName="discovery" Jul 08 18:11:15.063276 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.063260 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" Jul 08 18:11:15.065959 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.065942 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rp9sh\"" Jul 08 18:11:15.066987 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.066963 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 18:11:15.067046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.066963 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 18:11:15.070907 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.070825 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hf5q"] Jul 08 18:11:15.124775 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.124751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fch86\" (UniqueName: \"kubernetes.io/projected/e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c-kube-api-access-fch86\") pod \"kuadrant-operator-catalog-4hf5q\" (UID: \"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c\") " pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" Jul 08 18:11:15.128035 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.128006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:15.225537 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.225458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fch86\" (UniqueName: \"kubernetes.io/projected/e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c-kube-api-access-fch86\") pod \"kuadrant-operator-catalog-4hf5q\" (UID: \"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c\") " pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" Jul 08 18:11:15.236290 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.236263 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fch86\" (UniqueName: \"kubernetes.io/projected/e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c-kube-api-access-fch86\") pod \"kuadrant-operator-catalog-4hf5q\" (UID: \"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c\") " pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" Jul 08 18:11:15.373878 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.373823 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" Jul 08 18:11:15.411517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.411485 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hf5q"] Jul 08 18:11:15.511852 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.511767 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hf5q"] Jul 08 18:11:15.515349 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:11:15.515323 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c4b0df_0f3c_46d5_b2c5_5f2ce7cf776c.slice/crio-8c87d0daaf90643bcb39286fa32bc8c5e239cc8b2c7575030165ca3cfe49c3da WatchSource:0}: Error finding container 8c87d0daaf90643bcb39286fa32bc8c5e239cc8b2c7575030165ca3cfe49c3da: Status 404 returned error can't find the container with id 8c87d0daaf90643bcb39286fa32bc8c5e239cc8b2c7575030165ca3cfe49c3da Jul 08 18:11:15.621382 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.621352 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7qrwt"] Jul 08 18:11:15.626283 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.626268 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:15.634441 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.634411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7qrwt"] Jul 08 18:11:15.710455 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.710427 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" event={"ID":"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c","Type":"ContainerStarted","Data":"8c87d0daaf90643bcb39286fa32bc8c5e239cc8b2c7575030165ca3cfe49c3da"} Jul 08 18:11:15.730029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.730003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rkhf\" (UniqueName: \"kubernetes.io/projected/ba17c761-c27a-4018-a030-1264f90809e6-kube-api-access-9rkhf\") pod \"kuadrant-operator-catalog-7qrwt\" (UID: \"ba17c761-c27a-4018-a030-1264f90809e6\") " pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:15.830973 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.830900 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rkhf\" (UniqueName: \"kubernetes.io/projected/ba17c761-c27a-4018-a030-1264f90809e6-kube-api-access-9rkhf\") pod \"kuadrant-operator-catalog-7qrwt\" (UID: \"ba17c761-c27a-4018-a030-1264f90809e6\") " pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:15.839998 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.839976 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rkhf\" (UniqueName: \"kubernetes.io/projected/ba17c761-c27a-4018-a030-1264f90809e6-kube-api-access-9rkhf\") pod \"kuadrant-operator-catalog-7qrwt\" (UID: \"ba17c761-c27a-4018-a030-1264f90809e6\") " pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:15.936265 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:15.936223 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:16.072009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:16.071979 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7qrwt"] Jul 08 18:11:16.090196 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:11:16.090135 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba17c761_c27a_4018_a030_1264f90809e6.slice/crio-e42daca9194d74089133567ef122efd5f0ea7fba0bc1752acd2edc2929ea9338 WatchSource:0}: Error finding container e42daca9194d74089133567ef122efd5f0ea7fba0bc1752acd2edc2929ea9338: Status 404 returned error can't find the container with id e42daca9194d74089133567ef122efd5f0ea7fba0bc1752acd2edc2929ea9338 Jul 08 18:11:16.717991 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:16.717948 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" event={"ID":"ba17c761-c27a-4018-a030-1264f90809e6","Type":"ContainerStarted","Data":"e42daca9194d74089133567ef122efd5f0ea7fba0bc1752acd2edc2929ea9338"} Jul 08 18:11:18.729930 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:18.729893 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" event={"ID":"ba17c761-c27a-4018-a030-1264f90809e6","Type":"ContainerStarted","Data":"5f647a40823677640d181b2c5115faa0f6c3602cab0a7c73f00f564bb289923a"} Jul 08 18:11:18.731448 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:18.731421 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" event={"ID":"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c","Type":"ContainerStarted","Data":"d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe"} Jul 08 18:11:18.731559 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:18.731489 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" podUID="e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c" containerName="registry-server" containerID="cri-o://d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe" gracePeriod=2 Jul 08 18:11:18.753820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:18.753783 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" podStartSLOduration=1.648902962 podStartE2EDuration="3.753769941s" totalImagesPullingTime="2.104866979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:18.753509301 +0000 UTC m=+434.438716072" watchObservedRunningTime="2026-07-08 18:11:18.753769941 +0000 UTC m=+434.438976709" Jul 08 18:11:18.773609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:18.773568 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" podStartSLOduration=1.097893441 podStartE2EDuration="3.773554487s" totalImagesPullingTime="2.675661046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:18.770928309 +0000 UTC m=+434.456135077" watchObservedRunningTime="2026-07-08 18:11:18.773554487 +0000 UTC m=+434.458761256" Jul 08 18:11:18.976781 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:18.976758 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" Jul 08 18:11:19.058148 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.058079 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fch86\" (UniqueName: \"kubernetes.io/projected/e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c-kube-api-access-fch86\") pod \"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c\" (UID: \"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c\") " Jul 08 18:11:19.060011 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.059990 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c-kube-api-access-fch86" (OuterVolumeSpecName: "kube-api-access-fch86") pod "e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c" (UID: "e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c"). InnerVolumeSpecName "kube-api-access-fch86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:11:19.158684 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.158655 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fch86\" (UniqueName: \"kubernetes.io/projected/e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c-kube-api-access-fch86\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:11:19.737927 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.737896 2580 generic.go:356] "Generic (PLEG): container finished" podID="e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c" containerID="d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe" exitCode=0 Jul 08 18:11:19.738329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.737943 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" Jul 08 18:11:19.738329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.737982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" event={"ID":"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c","Type":"ContainerDied","Data":"d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe"} Jul 08 18:11:19.738329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.738027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4hf5q" event={"ID":"e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c","Type":"ContainerDied","Data":"8c87d0daaf90643bcb39286fa32bc8c5e239cc8b2c7575030165ca3cfe49c3da"} Jul 08 18:11:19.738329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.738049 2580 scope.go:117] "RemoveContainer" containerID="d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe" Jul 08 18:11:19.747194 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.747176 2580 scope.go:117] "RemoveContainer" containerID="d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe" Jul 08 18:11:19.747426 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:11:19.747406 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe\": container with ID starting with d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe not found: ID does not exist" containerID="d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe" Jul 08 18:11:19.747490 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.747434 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe"} err="failed to get container status \"d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe\": rpc error: code = NotFound desc = could not find container \"d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe\": container with ID starting with d0f77929efbeec4c925af1d717afaa1cf80af909b5cc4ae1e919ebbb84a713fe not found: ID does not exist" Jul 08 18:11:19.764818 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.764800 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hf5q"] Jul 08 18:11:19.768867 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:19.768848 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hf5q"] Jul 08 18:11:20.323087 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:20.323053 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:20.868230 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:20.868198 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c" path="/var/lib/kubelet/pods/e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c/volumes" Jul 08 18:11:25.936944 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:25.936912 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:25.937352 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:25.936991 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:25.958780 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:25.958759 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:26.790198 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:26.790169 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-7qrwt" Jul 08 18:11:44.065567 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:44.065534 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:47.481036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:47.480996 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:48.828160 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:48.828122 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct"] Jul 08 18:11:48.828627 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:48.828606 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c" containerName="registry-server" Jul 08 18:11:48.828707 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:48.828630 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c" containerName="registry-server" Jul 08 18:11:48.828760 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:48.828746 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5c4b0df-0f3c-46d5-b2c5-5f2ce7cf776c" containerName="registry-server" Jul 08 18:11:48.835132 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:48.835111 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:48.838728 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:48.838711 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gxlrs\"" Jul 08 18:11:48.847880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:48.847857 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct"] Jul 08 18:11:49.012182 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.012149 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/836a008f-927b-4383-9940-89d91ed6d05c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:49.012367 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.012215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mxm5\" (UniqueName: \"kubernetes.io/projected/836a008f-927b-4383-9940-89d91ed6d05c-kube-api-access-7mxm5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:49.113563 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.113485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/836a008f-927b-4383-9940-89d91ed6d05c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:49.113563 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.113542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mxm5\" (UniqueName: \"kubernetes.io/projected/836a008f-927b-4383-9940-89d91ed6d05c-kube-api-access-7mxm5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:49.113938 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.113915 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/836a008f-927b-4383-9940-89d91ed6d05c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:49.125388 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.125362 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mxm5\" (UniqueName: \"kubernetes.io/projected/836a008f-927b-4383-9940-89d91ed6d05c-kube-api-access-7mxm5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:49.144904 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.144884 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:49.287169 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.287141 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct"] Jul 08 18:11:49.289660 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:11:49.289634 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836a008f_927b_4383_9940_89d91ed6d05c.slice/crio-a5d992a04dfc66cfc3f86eeefd374cce2375f160462549ce49c0f64dacb48771 WatchSource:0}: Error finding container a5d992a04dfc66cfc3f86eeefd374cce2375f160462549ce49c0f64dacb48771: Status 404 returned error can't find the container with id a5d992a04dfc66cfc3f86eeefd374cce2375f160462549ce49c0f64dacb48771 Jul 08 18:11:49.461136 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.461102 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:49.868347 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:49.868264 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" event={"ID":"836a008f-927b-4383-9940-89d91ed6d05c","Type":"ContainerStarted","Data":"a5d992a04dfc66cfc3f86eeefd374cce2375f160462549ce49c0f64dacb48771"} Jul 08 18:11:50.850309 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:50.850276 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4"] Jul 08 18:11:50.853918 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:50.853897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" Jul 08 18:11:50.856657 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:50.856636 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 08 18:11:50.856657 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:50.856651 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2twcp\"" Jul 08 18:11:50.870933 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:50.870908 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4"] Jul 08 18:11:50.928823 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:50.928784 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jqwt\" (UniqueName: \"kubernetes.io/projected/e867ceca-ea46-45c0-9659-e81210d2a2de-kube-api-access-9jqwt\") pod \"dns-operator-controller-manager-864dd78c4c-mm2q4\" (UID: \"e867ceca-ea46-45c0-9659-e81210d2a2de\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" Jul 08 18:11:51.030140 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:51.030114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jqwt\" (UniqueName: \"kubernetes.io/projected/e867ceca-ea46-45c0-9659-e81210d2a2de-kube-api-access-9jqwt\") pod \"dns-operator-controller-manager-864dd78c4c-mm2q4\" (UID: \"e867ceca-ea46-45c0-9659-e81210d2a2de\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" Jul 08 18:11:51.049295 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:51.049264 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jqwt\" (UniqueName: \"kubernetes.io/projected/e867ceca-ea46-45c0-9659-e81210d2a2de-kube-api-access-9jqwt\") pod \"dns-operator-controller-manager-864dd78c4c-mm2q4\" (UID: \"e867ceca-ea46-45c0-9659-e81210d2a2de\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" Jul 08 18:11:51.166225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:51.166176 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" Jul 08 18:11:51.318889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:51.318864 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4"] Jul 08 18:11:51.321444 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:11:51.321416 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode867ceca_ea46_45c0_9659_e81210d2a2de.slice/crio-df1ff8f23d9a041d53419b074c5db16072217fbd5623694c1420fa41d7a0a29f WatchSource:0}: Error finding container df1ff8f23d9a041d53419b074c5db16072217fbd5623694c1420fa41d7a0a29f: Status 404 returned error can't find the container with id df1ff8f23d9a041d53419b074c5db16072217fbd5623694c1420fa41d7a0a29f Jul 08 18:11:51.411504 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:51.411468 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:51.510552 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:51.510469 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:51.880817 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:51.880751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" event={"ID":"e867ceca-ea46-45c0-9659-e81210d2a2de","Type":"ContainerStarted","Data":"df1ff8f23d9a041d53419b074c5db16072217fbd5623694c1420fa41d7a0a29f"} Jul 08 18:11:53.731094 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.731062 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9"] Jul 08 18:11:53.734994 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.734963 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:11:53.740335 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.740309 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kt89x\"" Jul 08 18:11:53.752088 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.752015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8lxs\" (UniqueName: \"kubernetes.io/projected/358f682e-0ca3-40ab-a398-8d61df3af89d-kube-api-access-m8lxs\") pod \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" (UID: \"358f682e-0ca3-40ab-a398-8d61df3af89d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:11:53.752430 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.752401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:53.754746 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.754720 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9"] Jul 08 18:11:53.852932 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.852899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8lxs\" (UniqueName: \"kubernetes.io/projected/358f682e-0ca3-40ab-a398-8d61df3af89d-kube-api-access-m8lxs\") pod \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" (UID: \"358f682e-0ca3-40ab-a398-8d61df3af89d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:11:53.867495 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.867461 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8lxs\" (UniqueName: \"kubernetes.io/projected/358f682e-0ca3-40ab-a398-8d61df3af89d-kube-api-access-m8lxs\") pod \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" (UID: \"358f682e-0ca3-40ab-a398-8d61df3af89d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:11:53.911335 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:53.911276 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:54.048353 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:54.048286 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:11:54.203992 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:54.203971 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9"] Jul 08 18:11:54.206334 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:11:54.206312 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358f682e_0ca3_40ab_a398_8d61df3af89d.slice/crio-a28498bc17f2ac5a4d8c3575f985196a5933a3db6f55fb2d641bbb61da5bbc8e WatchSource:0}: Error finding container a28498bc17f2ac5a4d8c3575f985196a5933a3db6f55fb2d641bbb61da5bbc8e: Status 404 returned error can't find the container with id a28498bc17f2ac5a4d8c3575f985196a5933a3db6f55fb2d641bbb61da5bbc8e Jul 08 18:11:54.643352 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:54.643288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:54.898115 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:54.898015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" event={"ID":"358f682e-0ca3-40ab-a398-8d61df3af89d","Type":"ContainerStarted","Data":"a28498bc17f2ac5a4d8c3575f985196a5933a3db6f55fb2d641bbb61da5bbc8e"} Jul 08 18:11:54.900096 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:54.900055 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" event={"ID":"e867ceca-ea46-45c0-9659-e81210d2a2de","Type":"ContainerStarted","Data":"0bae5327d1a4471efe8f25863550868120eecded4264e249b2841dcee8022488"} Jul 08 18:11:54.900240 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:54.900189 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" Jul 08 18:11:54.939591 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:54.939542 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" podStartSLOduration=2.168583639 podStartE2EDuration="4.939520249s" totalImagesPullingTime="2.77093661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:54.936094908 +0000 UTC m=+470.621301678" watchObservedRunningTime="2026-07-08 18:11:54.939520249 +0000 UTC m=+470.624727018" Jul 08 18:11:56.077275 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:56.077231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:57.236794 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.236772 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wnblj"] Jul 08 18:11:57.242413 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.242392 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" Jul 08 18:11:57.248377 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.248355 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-579rt\"" Jul 08 18:11:57.252579 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.252551 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wnblj"] Jul 08 18:11:57.285588 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.285558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs246\" (UniqueName: \"kubernetes.io/projected/8d39a9bc-ec97-4066-93db-c55f9890c05a-kube-api-access-vs246\") pod \"authorino-operator-75cc8d5cb7-wnblj\" (UID: \"8d39a9bc-ec97-4066-93db-c55f9890c05a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" Jul 08 18:11:57.386210 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.386183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs246\" (UniqueName: \"kubernetes.io/projected/8d39a9bc-ec97-4066-93db-c55f9890c05a-kube-api-access-vs246\") pod \"authorino-operator-75cc8d5cb7-wnblj\" (UID: \"8d39a9bc-ec97-4066-93db-c55f9890c05a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" Jul 08 18:11:57.395741 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.395710 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs246\" (UniqueName: \"kubernetes.io/projected/8d39a9bc-ec97-4066-93db-c55f9890c05a-kube-api-access-vs246\") pod \"authorino-operator-75cc8d5cb7-wnblj\" (UID: \"8d39a9bc-ec97-4066-93db-c55f9890c05a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" Jul 08 18:11:57.642035 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.641998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" Jul 08 18:11:57.778706 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.778684 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wnblj"] Jul 08 18:11:57.781226 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:11:57.781196 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d39a9bc_ec97_4066_93db_c55f9890c05a.slice/crio-f2d63e5db72b365f5c34a7c9d777c3ccbfb1e9d20e1931eac82b0eccd6a80ec4 WatchSource:0}: Error finding container f2d63e5db72b365f5c34a7c9d777c3ccbfb1e9d20e1931eac82b0eccd6a80ec4: Status 404 returned error can't find the container with id f2d63e5db72b365f5c34a7c9d777c3ccbfb1e9d20e1931eac82b0eccd6a80ec4 Jul 08 18:11:57.918488 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.918405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" event={"ID":"836a008f-927b-4383-9940-89d91ed6d05c","Type":"ContainerStarted","Data":"e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d"} Jul 08 18:11:57.918638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.918501 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:11:57.920013 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.919984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" event={"ID":"358f682e-0ca3-40ab-a398-8d61df3af89d","Type":"ContainerStarted","Data":"946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db"} Jul 08 18:11:57.920133 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.920079 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:11:57.921232 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.921206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" event={"ID":"8d39a9bc-ec97-4066-93db-c55f9890c05a","Type":"ContainerStarted","Data":"f2d63e5db72b365f5c34a7c9d777c3ccbfb1e9d20e1931eac82b0eccd6a80ec4"} Jul 08 18:11:57.939533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.939480 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" podStartSLOduration=2.008623506 podStartE2EDuration="9.939467457s" totalImagesPullingTime="7.930843951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:57.938284933 +0000 UTC m=+473.623491701" watchObservedRunningTime="2026-07-08 18:11:57.939467457 +0000 UTC m=+473.624674279" Jul 08 18:11:57.961413 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:57.961371 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" podStartSLOduration=1.9539844130000001 podStartE2EDuration="4.961358694s" totalImagesPullingTime="3.007374281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:57.959651039 +0000 UTC m=+473.644857808" watchObservedRunningTime="2026-07-08 18:11:57.961358694 +0000 UTC m=+473.646565462" Jul 08 18:11:59.061125 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:59.061081 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:11:59.167416 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:11:59.167375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:00.260170 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:00.260126 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:00.598438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:00.598337 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:00.940791 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:00.940761 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" event={"ID":"8d39a9bc-ec97-4066-93db-c55f9890c05a","Type":"ContainerStarted","Data":"30615c8216a66aa76b15c628cfe6d401fb8e509a465cebe2c096b8a9e8f9e40f"} Jul 08 18:12:00.940979 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:00.940964 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" Jul 08 18:12:00.961940 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:00.961894 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" podStartSLOduration=1.4590223 podStartE2EDuration="3.961879419s" totalImagesPullingTime="2.502857119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:12:00.960149178 +0000 UTC m=+476.645355947" watchObservedRunningTime="2026-07-08 18:12:00.961879419 +0000 UTC m=+476.647086396" Jul 08 18:12:05.909002 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:05.908971 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mm2q4" Jul 08 18:12:06.766120 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:06.766092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:06.892124 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:06.892090 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:08.929627 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:08.929599 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:08.930024 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:08.929664 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:10.892640 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.892608 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct"] Jul 08 18:12:10.893069 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.892802 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" podUID="836a008f-927b-4383-9940-89d91ed6d05c" containerName="manager" containerID="cri-o://e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d" gracePeriod=2 Jul 08 18:12:10.898549 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.898521 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct"] Jul 08 18:12:10.921380 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.921357 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9"] Jul 08 18:12:10.921575 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.921538 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" containerName="manager" containerID="cri-o://946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db" gracePeriod=2 Jul 08 18:12:10.925399 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:10.925369 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:10.935273 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.935245 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k"] Jul 08 18:12:10.935683 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.935666 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="836a008f-927b-4383-9940-89d91ed6d05c" containerName="manager" Jul 08 18:12:10.935683 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.935684 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="836a008f-927b-4383-9940-89d91ed6d05c" containerName="manager" Jul 08 18:12:10.935821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.935777 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="836a008f-927b-4383-9940-89d91ed6d05c" containerName="manager" Jul 08 18:12:10.938927 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.938905 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9"] Jul 08 18:12:10.939052 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.939034 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:10.949295 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:10.949263 2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"358f682e-0ca3-40ab-a398-8d61df3af89d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-tlbq9\": pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:10.952899 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:10.952874 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:10.955011 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.954988 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k"] Jul 08 18:12:10.957733 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:10.957710 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:10.963889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.963867 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz"] Jul 08 18:12:10.964230 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.964218 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" containerName="manager" Jul 08 18:12:10.964272 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.964233 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" containerName="manager" Jul 08 18:12:10.964305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.964296 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" containerName="manager" Jul 08 18:12:10.967194 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.967179 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" Jul 08 18:12:10.986698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:10.986677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz"] Jul 08 18:12:10.990061 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:10.990030 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:10.992126 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:10.992096 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:11.002891 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.002864 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzknn\" (UniqueName: \"kubernetes.io/projected/e7ca7d89-cf72-40eb-9241-7b769750730e-kube-api-access-nzknn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrw5k\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:11.003005 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.002921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7ca7d89-cf72-40eb-9241-7b769750730e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrw5k\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:11.103530 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.103493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56t8s\" (UniqueName: \"kubernetes.io/projected/d6fdf493-475a-4184-8ad8-1077c89ecb38-kube-api-access-56t8s\") pod \"limitador-operator-controller-manager-859d7b78bd-cr2rz\" (UID: \"d6fdf493-475a-4184-8ad8-1077c89ecb38\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" Jul 08 18:12:11.104029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.103998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzknn\" (UniqueName: \"kubernetes.io/projected/e7ca7d89-cf72-40eb-9241-7b769750730e-kube-api-access-nzknn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrw5k\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:11.104135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.104065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7ca7d89-cf72-40eb-9241-7b769750730e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrw5k\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:11.104591 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.104565 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7ca7d89-cf72-40eb-9241-7b769750730e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrw5k\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:11.114780 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.114747 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzknn\" (UniqueName: \"kubernetes.io/projected/e7ca7d89-cf72-40eb-9241-7b769750730e-kube-api-access-nzknn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrw5k\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:11.162752 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.162726 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:11.165493 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.165473 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:11.166533 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:11.166512 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:11.172611 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:11.172587 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:11.177451 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:11.177429 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:11.182313 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:11.182280 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:11.204669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.204644 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8lxs\" (UniqueName: \"kubernetes.io/projected/358f682e-0ca3-40ab-a398-8d61df3af89d-kube-api-access-m8lxs\") pod \"358f682e-0ca3-40ab-a398-8d61df3af89d\" (UID: \"358f682e-0ca3-40ab-a398-8d61df3af89d\") " Jul 08 18:12:11.204761 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.204747 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/836a008f-927b-4383-9940-89d91ed6d05c-extensions-socket-volume\") pod \"836a008f-927b-4383-9940-89d91ed6d05c\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " Jul 08 18:12:11.204822 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.204808 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mxm5\" (UniqueName: \"kubernetes.io/projected/836a008f-927b-4383-9940-89d91ed6d05c-kube-api-access-7mxm5\") pod \"836a008f-927b-4383-9940-89d91ed6d05c\" (UID: \"836a008f-927b-4383-9940-89d91ed6d05c\") " Jul 08 18:12:11.205008 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.204988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56t8s\" (UniqueName: \"kubernetes.io/projected/d6fdf493-475a-4184-8ad8-1077c89ecb38-kube-api-access-56t8s\") pod \"limitador-operator-controller-manager-859d7b78bd-cr2rz\" (UID: \"d6fdf493-475a-4184-8ad8-1077c89ecb38\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" Jul 08 18:12:11.205148 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.205124 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/836a008f-927b-4383-9940-89d91ed6d05c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "836a008f-927b-4383-9940-89d91ed6d05c" (UID: "836a008f-927b-4383-9940-89d91ed6d05c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:11.206761 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.206736 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358f682e-0ca3-40ab-a398-8d61df3af89d-kube-api-access-m8lxs" (OuterVolumeSpecName: "kube-api-access-m8lxs") pod "358f682e-0ca3-40ab-a398-8d61df3af89d" (UID: "358f682e-0ca3-40ab-a398-8d61df3af89d"). InnerVolumeSpecName "kube-api-access-m8lxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:12:11.206827 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.206768 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/836a008f-927b-4383-9940-89d91ed6d05c-kube-api-access-7mxm5" (OuterVolumeSpecName: "kube-api-access-7mxm5") pod "836a008f-927b-4383-9940-89d91ed6d05c" (UID: "836a008f-927b-4383-9940-89d91ed6d05c"). InnerVolumeSpecName "kube-api-access-7mxm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:12:11.216518 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.216495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56t8s\" (UniqueName: \"kubernetes.io/projected/d6fdf493-475a-4184-8ad8-1077c89ecb38-kube-api-access-56t8s\") pod \"limitador-operator-controller-manager-859d7b78bd-cr2rz\" (UID: \"d6fdf493-475a-4184-8ad8-1077c89ecb38\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" Jul 08 18:12:11.258695 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.258665 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:11.305899 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.305874 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8lxs\" (UniqueName: \"kubernetes.io/projected/358f682e-0ca3-40ab-a398-8d61df3af89d-kube-api-access-m8lxs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:12:11.305899 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.305897 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/836a008f-927b-4383-9940-89d91ed6d05c-extensions-socket-volume\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:12:11.306042 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.305906 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mxm5\" (UniqueName: \"kubernetes.io/projected/836a008f-927b-4383-9940-89d91ed6d05c-kube-api-access-7mxm5\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:12:11.321165 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.321145 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:11.327266 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.327251 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" Jul 08 18:12:11.497472 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.497338 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k"] Jul 08 18:12:11.497821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.497788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:11.501819 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:12:11.501788 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7ca7d89_cf72_40eb_9241_7b769750730e.slice/crio-ac488b5c4d060a0d31c615ace7b985d5ffb211cafa9f18d69e9b4a28a6390703 WatchSource:0}: Error finding container ac488b5c4d060a0d31c615ace7b985d5ffb211cafa9f18d69e9b4a28a6390703: Status 404 returned error can't find the container with id ac488b5c4d060a0d31c615ace7b985d5ffb211cafa9f18d69e9b4a28a6390703 Jul 08 18:12:11.527438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.527411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz"] Jul 08 18:12:11.531891 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:12:11.531869 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fdf493_475a_4184_8ad8_1077c89ecb38.slice/crio-de9d23af1b1a73e82934f1ed450a887cecc1951409dd782c42e959b01b9ff587 WatchSource:0}: Error finding container de9d23af1b1a73e82934f1ed450a887cecc1951409dd782c42e959b01b9ff587: Status 404 returned error can't find the container with id de9d23af1b1a73e82934f1ed450a887cecc1951409dd782c42e959b01b9ff587 Jul 08 18:12:11.623185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.623156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:11.947605 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.947576 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wnblj" Jul 08 18:12:11.989024 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:11.988983 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:11.991436 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:11.991412 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:11.996216 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.996194 2580 generic.go:356] "Generic (PLEG): container finished" podID="836a008f-927b-4383-9940-89d91ed6d05c" containerID="e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d" exitCode=0 Jul 08 18:12:11.996310 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.996229 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:11.996310 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.996253 2580 scope.go:117] "RemoveContainer" containerID="e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d" Jul 08 18:12:11.997678 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.997662 2580 generic.go:356] "Generic (PLEG): container finished" podID="358f682e-0ca3-40ab-a398-8d61df3af89d" containerID="946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db" exitCode=0 Jul 08 18:12:11.997742 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.997720 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:11.999264 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:11.999163 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:11.999563 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.999503 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" event={"ID":"d6fdf493-475a-4184-8ad8-1077c89ecb38","Type":"ContainerStarted","Data":"17a041d367b867c6e99d5c5d70b0ce6b3c3c1a2edbcdc83d584187e6f90acfa1"} Jul 08 18:12:11.999563 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.999552 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" event={"ID":"d6fdf493-475a-4184-8ad8-1077c89ecb38","Type":"ContainerStarted","Data":"de9d23af1b1a73e82934f1ed450a887cecc1951409dd782c42e959b01b9ff587"} Jul 08 18:12:11.999703 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:11.999580 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" Jul 08 18:12:12.001265 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.001215 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" event={"ID":"e7ca7d89-cf72-40eb-9241-7b769750730e","Type":"ContainerStarted","Data":"949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6"} Jul 08 18:12:12.001265 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.001259 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" event={"ID":"e7ca7d89-cf72-40eb-9241-7b769750730e","Type":"ContainerStarted","Data":"ac488b5c4d060a0d31c615ace7b985d5ffb211cafa9f18d69e9b4a28a6390703"} Jul 08 18:12:12.001442 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.001289 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:12.002635 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:12.002605 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:12.007439 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.007347 2580 scope.go:117] "RemoveContainer" containerID="e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d" Jul 08 18:12:12.007671 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:12.007648 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d\": container with ID starting with e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d not found: ID does not exist" containerID="e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d" Jul 08 18:12:12.007753 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.007678 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d"} err="failed to get container status \"e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d\": rpc error: code = NotFound desc = could not find container \"e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d\": container with ID starting with e038d20f11fd8883bc73666b96ee484aae9554b5023200e54577db8102da0d4d not found: ID does not exist" Jul 08 18:12:12.007753 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.007694 2580 scope.go:117] "RemoveContainer" containerID="946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db" Jul 08 18:12:12.017779 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.017756 2580 scope.go:117] "RemoveContainer" containerID="946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db" Jul 08 18:12:12.018138 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:12.018116 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db\": container with ID starting with 946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db not found: ID does not exist" containerID="946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db" Jul 08 18:12:12.018232 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.018147 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db"} err="failed to get container status \"946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db\": rpc error: code = NotFound desc = could not find container \"946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db\": container with ID starting with 946079d838b9a3a601393f5e161db131bd2673c670c1ffeeb9b39cd37ae800db not found: ID does not exist" Jul 08 18:12:12.034196 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.033176 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" podStartSLOduration=2.033156768 podStartE2EDuration="2.033156768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:12:12.032457352 +0000 UTC m=+487.717664121" watchObservedRunningTime="2026-07-08 18:12:12.033156768 +0000 UTC m=+487.718363538" Jul 08 18:12:12.067862 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.067778 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" podStartSLOduration=2.067762112 podStartE2EDuration="2.067762112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:12:12.066424548 +0000 UTC m=+487.751631310" watchObservedRunningTime="2026-07-08 18:12:12.067762112 +0000 UTC m=+487.752968880" Jul 08 18:12:12.069176 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:12.069152 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:12.072374 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:12.072351 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:12.076096 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:12.076077 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pdwct\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="836a008f-927b-4383-9940-89d91ed6d05c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pdwct" Jul 08 18:12:12.079093 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:12.079072 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlbq9\" is forbidden: User \"system:node:ip-10-0-141-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-56.ec2.internal' and this object" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlbq9" Jul 08 18:12:12.869869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.869812 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="358f682e-0ca3-40ab-a398-8d61df3af89d" path="/var/lib/kubelet/pods/358f682e-0ca3-40ab-a398-8d61df3af89d/volumes" Jul 08 18:12:12.870190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:12.870177 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="836a008f-927b-4383-9940-89d91ed6d05c" path="/var/lib/kubelet/pods/836a008f-927b-4383-9940-89d91ed6d05c/volumes" Jul 08 18:12:16.323812 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:16.323775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:16.452805 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:16.452766 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:16.564029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:16.563984 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:20.280057 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:20.280025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:20.401259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:20.401224 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:23.012485 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:23.012459 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr2rz" Jul 08 18:12:23.012872 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:23.012523 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:28.268441 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:28.268407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:28.370889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:28.370851 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:31.457755 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.457714 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k"] Jul 08 18:12:31.458254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.457962 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" podUID="e7ca7d89-cf72-40eb-9241-7b769750730e" containerName="manager" containerID="cri-o://949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6" gracePeriod=10 Jul 08 18:12:31.719175 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.719111 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:31.876242 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.876211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:31.885149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.885127 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7ca7d89-cf72-40eb-9241-7b769750730e-extensions-socket-volume\") pod \"e7ca7d89-cf72-40eb-9241-7b769750730e\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " Jul 08 18:12:31.885235 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.885163 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzknn\" (UniqueName: \"kubernetes.io/projected/e7ca7d89-cf72-40eb-9241-7b769750730e-kube-api-access-nzknn\") pod \"e7ca7d89-cf72-40eb-9241-7b769750730e\" (UID: \"e7ca7d89-cf72-40eb-9241-7b769750730e\") " Jul 08 18:12:31.885504 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.885483 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ca7d89-cf72-40eb-9241-7b769750730e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7ca7d89-cf72-40eb-9241-7b769750730e" (UID: "e7ca7d89-cf72-40eb-9241-7b769750730e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:12:31.887259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.887238 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7ca7d89-cf72-40eb-9241-7b769750730e-kube-api-access-nzknn" (OuterVolumeSpecName: "kube-api-access-nzknn") pod "e7ca7d89-cf72-40eb-9241-7b769750730e" (UID: "e7ca7d89-cf72-40eb-9241-7b769750730e"). InnerVolumeSpecName "kube-api-access-nzknn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:12:31.985735 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.985657 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7ca7d89-cf72-40eb-9241-7b769750730e-extensions-socket-volume\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:12:31.985735 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.985683 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzknn\" (UniqueName: \"kubernetes.io/projected/e7ca7d89-cf72-40eb-9241-7b769750730e-kube-api-access-nzknn\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:12:31.991948 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:31.991906 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:32.098895 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.098864 2580 generic.go:356] "Generic (PLEG): container finished" podID="e7ca7d89-cf72-40eb-9241-7b769750730e" containerID="949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6" exitCode=0 Jul 08 18:12:32.099046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.098899 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" Jul 08 18:12:32.099046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.098944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" event={"ID":"e7ca7d89-cf72-40eb-9241-7b769750730e","Type":"ContainerDied","Data":"949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6"} Jul 08 18:12:32.099046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.098995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k" event={"ID":"e7ca7d89-cf72-40eb-9241-7b769750730e","Type":"ContainerDied","Data":"ac488b5c4d060a0d31c615ace7b985d5ffb211cafa9f18d69e9b4a28a6390703"} Jul 08 18:12:32.099046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.099018 2580 scope.go:117] "RemoveContainer" containerID="949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6" Jul 08 18:12:32.108259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.108245 2580 scope.go:117] "RemoveContainer" containerID="949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6" Jul 08 18:12:32.108501 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:12:32.108483 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6\": container with ID starting with 949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6 not found: ID does not exist" containerID="949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6" Jul 08 18:12:32.108552 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.108508 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6"} err="failed to get container status \"949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6\": rpc error: code = NotFound desc = could not find container \"949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6\": container with ID starting with 949a64447e538337a774623a75df7a1974ffbf2880156e319a3ff9deb2388dd6 not found: ID does not exist" Jul 08 18:12:32.129909 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.129880 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k"] Jul 08 18:12:32.138318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.138282 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrw5k"] Jul 08 18:12:32.481319 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.481287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:32.587397 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.587366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:32.868803 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:32.868726 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7ca7d89-cf72-40eb-9241-7b769750730e" path="/var/lib/kubelet/pods/e7ca7d89-cf72-40eb-9241-7b769750730e/volumes" Jul 08 18:12:36.872527 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:36.872494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:36.973815 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:36.973765 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:37.256104 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:37.256068 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:47.993679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:47.993642 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68"] Jul 08 18:12:47.994074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:47.994059 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7ca7d89-cf72-40eb-9241-7b769750730e" containerName="manager" Jul 08 18:12:47.994074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:47.994075 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7ca7d89-cf72-40eb-9241-7b769750730e" containerName="manager" Jul 08 18:12:47.994160 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:47.994149 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7ca7d89-cf72-40eb-9241-7b769750730e" containerName="manager" Jul 08 18:12:47.997286 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:47.997270 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.000368 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.000350 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jz2r7\"" Jul 08 18:12:48.015853 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.015812 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59a21420-9066-4247-939c-aef787050364-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.015967 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.015881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.015967 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.015929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a21420-9066-4247-939c-aef787050364-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.015967 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.015954 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jrp6\" (UniqueName: \"kubernetes.io/projected/59a21420-9066-4247-939c-aef787050364-kube-api-access-5jrp6\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.016111 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.016046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.016209 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.016192 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.016279 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.016240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.016279 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.016265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59a21420-9066-4247-939c-aef787050364-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.016372 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.016311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.017784 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.017757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68"] Jul 08 18:12:48.030936 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.030908 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:48.116886 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.116854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.116886 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.116887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117081 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.116903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59a21420-9066-4247-939c-aef787050364-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117081 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117081 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59a21420-9066-4247-939c-aef787050364-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a21420-9066-4247-939c-aef787050364-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jrp6\" (UniqueName: \"kubernetes.io/projected/59a21420-9066-4247-939c-aef787050364-kube-api-access-5jrp6\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117234 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117242 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117501 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117501 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117489 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117566 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.117850 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.117807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59a21420-9066-4247-939c-aef787050364-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.119135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.119109 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59a21420-9066-4247-939c-aef787050364-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.119743 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.119722 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59a21420-9066-4247-939c-aef787050364-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.129701 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.129674 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jrp6\" (UniqueName: \"kubernetes.io/projected/59a21420-9066-4247-939c-aef787050364-kube-api-access-5jrp6\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.130870 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.130850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59a21420-9066-4247-939c-aef787050364-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vfz68\" (UID: \"59a21420-9066-4247-939c-aef787050364\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.307530 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.307453 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:48.459417 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.459393 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68"] Jul 08 18:12:48.461187 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:12:48.461165 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a21420_9066_4247_939c_aef787050364.slice/crio-7feb777cb0f23e77d1eb76ceca2cc55223f84ffda02ea2831cb094a49555b325 WatchSource:0}: Error finding container 7feb777cb0f23e77d1eb76ceca2cc55223f84ffda02ea2831cb094a49555b325: Status 404 returned error can't find the container with id 7feb777cb0f23e77d1eb76ceca2cc55223f84ffda02ea2831cb094a49555b325 Jul 08 18:12:48.463417 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.463384 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:12:48.463498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.463451 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:12:48.463498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.463481 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:12:48.671807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:48.671765 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:49.181886 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:49.181855 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" event={"ID":"59a21420-9066-4247-939c-aef787050364","Type":"ContainerStarted","Data":"16803c63eb99f0a01480050285f07c719fd96ae50c81fbc97d0de58545f65654"} Jul 08 18:12:49.182251 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:49.181903 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" event={"ID":"59a21420-9066-4247-939c-aef787050364","Type":"ContainerStarted","Data":"7feb777cb0f23e77d1eb76ceca2cc55223f84ffda02ea2831cb094a49555b325"} Jul 08 18:12:49.217084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:49.217029 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" podStartSLOduration=2.217008709 podStartE2EDuration="2.217008709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:12:49.215005115 +0000 UTC m=+524.900211885" watchObservedRunningTime="2026-07-08 18:12:49.217008709 +0000 UTC m=+524.902215479" Jul 08 18:12:49.308353 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:49.308311 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:49.313534 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:49.313506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:50.186290 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:50.186260 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:50.187273 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:50.187255 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:12:52.869819 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.869784 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ds29x"] Jul 08 18:12:52.873121 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.873101 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:52.878631 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.878609 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6gbkt\"" Jul 08 18:12:52.878759 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.878610 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 18:12:52.882058 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.881047 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:52.905145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.905117 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ds29x"] Jul 08 18:12:52.956205 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.956175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d194cf3a-b6a0-4467-9061-6506b1a54443-config-file\") pod \"limitador-limitador-64c94fd64f-ds29x\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:52.956342 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.956255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h96m7\" (UniqueName: \"kubernetes.io/projected/d194cf3a-b6a0-4467-9061-6506b1a54443-kube-api-access-h96m7\") pod \"limitador-limitador-64c94fd64f-ds29x\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:52.973360 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:52.973324 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ds29x"] Jul 08 18:12:53.007188 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.007154 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:53.056994 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.056962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h96m7\" (UniqueName: \"kubernetes.io/projected/d194cf3a-b6a0-4467-9061-6506b1a54443-kube-api-access-h96m7\") pod \"limitador-limitador-64c94fd64f-ds29x\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:53.057159 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.057032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d194cf3a-b6a0-4467-9061-6506b1a54443-config-file\") pod \"limitador-limitador-64c94fd64f-ds29x\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:53.057634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.057611 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d194cf3a-b6a0-4467-9061-6506b1a54443-config-file\") pod \"limitador-limitador-64c94fd64f-ds29x\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:53.070683 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.070660 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h96m7\" (UniqueName: \"kubernetes.io/projected/d194cf3a-b6a0-4467-9061-6506b1a54443-kube-api-access-h96m7\") pod \"limitador-limitador-64c94fd64f-ds29x\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:53.111040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.111003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:53.189291 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.189265 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:53.343938 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.343918 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ds29x"] Jul 08 18:12:53.345516 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:12:53.345487 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd194cf3a_b6a0_4467_9061_6506b1a54443.slice/crio-8bf120ce87d57202bc28739546627737b6550a3026ec7798f421703a3d47b9f0 WatchSource:0}: Error finding container 8bf120ce87d57202bc28739546627737b6550a3026ec7798f421703a3d47b9f0: Status 404 returned error can't find the container with id 8bf120ce87d57202bc28739546627737b6550a3026ec7798f421703a3d47b9f0 Jul 08 18:12:53.872846 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.872815 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-82wb4"] Jul 08 18:12:53.877241 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.877224 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-82wb4" Jul 08 18:12:53.882333 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.882315 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7cf7r\"" Jul 08 18:12:53.901073 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.901051 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-82wb4"] Jul 08 18:12:53.928957 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.928922 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:53.964924 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:53.964901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpbz5\" (UniqueName: \"kubernetes.io/projected/1c8c3723-97b2-4e42-a61d-436b50b71167-kube-api-access-zpbz5\") pod \"authorino-58cd955675-82wb4\" (UID: \"1c8c3723-97b2-4e42-a61d-436b50b71167\") " pod="kuadrant-system/authorino-58cd955675-82wb4" Jul 08 18:12:54.033748 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:54.033713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:12:54.065497 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:54.065476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpbz5\" (UniqueName: \"kubernetes.io/projected/1c8c3723-97b2-4e42-a61d-436b50b71167-kube-api-access-zpbz5\") pod \"authorino-58cd955675-82wb4\" (UID: \"1c8c3723-97b2-4e42-a61d-436b50b71167\") " pod="kuadrant-system/authorino-58cd955675-82wb4" Jul 08 18:12:54.093435 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:54.093408 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpbz5\" (UniqueName: \"kubernetes.io/projected/1c8c3723-97b2-4e42-a61d-436b50b71167-kube-api-access-zpbz5\") pod \"authorino-58cd955675-82wb4\" (UID: \"1c8c3723-97b2-4e42-a61d-436b50b71167\") " pod="kuadrant-system/authorino-58cd955675-82wb4" Jul 08 18:12:54.186371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:54.186346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-82wb4" Jul 08 18:12:54.210880 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:54.210816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" event={"ID":"d194cf3a-b6a0-4467-9061-6506b1a54443","Type":"ContainerStarted","Data":"8bf120ce87d57202bc28739546627737b6550a3026ec7798f421703a3d47b9f0"} Jul 08 18:12:54.367033 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:54.366994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-82wb4"] Jul 08 18:12:54.371401 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:12:54.371370 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c8c3723_97b2_4e42_a61d_436b50b71167.slice/crio-b3077576a91d36027f18f062e0a1ea252ea562ddfaf8d4f820b185de21f6c839 WatchSource:0}: Error finding container b3077576a91d36027f18f062e0a1ea252ea562ddfaf8d4f820b185de21f6c839: Status 404 returned error can't find the container with id b3077576a91d36027f18f062e0a1ea252ea562ddfaf8d4f820b185de21f6c839 Jul 08 18:12:55.218493 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:55.218463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-82wb4" event={"ID":"1c8c3723-97b2-4e42-a61d-436b50b71167","Type":"ContainerStarted","Data":"b3077576a91d36027f18f062e0a1ea252ea562ddfaf8d4f820b185de21f6c839"} Jul 08 18:12:57.231409 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:57.231318 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" event={"ID":"d194cf3a-b6a0-4467-9061-6506b1a54443","Type":"ContainerStarted","Data":"54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e"} Jul 08 18:12:57.231868 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:57.231457 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:12:57.260986 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:12:57.260930 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" podStartSLOduration=2.218740256 podStartE2EDuration="5.260913784s" totalImagesPullingTime="3.042173528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:12:57.259628526 +0000 UTC m=+532.944835298" watchObservedRunningTime="2026-07-08 18:12:57.260913784 +0000 UTC m=+532.946120589" Jul 08 18:13:00.248097 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:00.248066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-82wb4" event={"ID":"1c8c3723-97b2-4e42-a61d-436b50b71167","Type":"ContainerStarted","Data":"decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900"} Jul 08 18:13:00.268407 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:00.268364 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-82wb4" podStartSLOduration=2.2343765270000002 podStartE2EDuration="7.26834638s" totalImagesPullingTime="5.033969853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:13:00.265535002 +0000 UTC m=+535.950741770" watchObservedRunningTime="2026-07-08 18:13:00.26834638 +0000 UTC m=+535.953553143" Jul 08 18:13:00.456812 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:00.456771 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-82wb4"] Jul 08 18:13:02.257926 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:02.257882 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-82wb4" podUID="1c8c3723-97b2-4e42-a61d-436b50b71167" containerName="authorino" containerID="cri-o://decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900" gracePeriod=30 Jul 08 18:13:02.505115 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:02.505092 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-82wb4" Jul 08 18:13:02.549675 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:02.549603 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpbz5\" (UniqueName: \"kubernetes.io/projected/1c8c3723-97b2-4e42-a61d-436b50b71167-kube-api-access-zpbz5\") pod \"1c8c3723-97b2-4e42-a61d-436b50b71167\" (UID: \"1c8c3723-97b2-4e42-a61d-436b50b71167\") " Jul 08 18:13:02.551628 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:02.551591 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c8c3723-97b2-4e42-a61d-436b50b71167-kube-api-access-zpbz5" (OuterVolumeSpecName: "kube-api-access-zpbz5") pod "1c8c3723-97b2-4e42-a61d-436b50b71167" (UID: "1c8c3723-97b2-4e42-a61d-436b50b71167"). InnerVolumeSpecName "kube-api-access-zpbz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:13:02.650520 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:02.650476 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpbz5\" (UniqueName: \"kubernetes.io/projected/1c8c3723-97b2-4e42-a61d-436b50b71167-kube-api-access-zpbz5\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:13:03.262995 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.262955 2580 generic.go:356] "Generic (PLEG): container finished" podID="1c8c3723-97b2-4e42-a61d-436b50b71167" containerID="decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900" exitCode=0 Jul 08 18:13:03.262995 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.262999 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-82wb4" Jul 08 18:13:03.263464 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.263000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-82wb4" event={"ID":"1c8c3723-97b2-4e42-a61d-436b50b71167","Type":"ContainerDied","Data":"decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900"} Jul 08 18:13:03.263464 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.263095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-82wb4" event={"ID":"1c8c3723-97b2-4e42-a61d-436b50b71167","Type":"ContainerDied","Data":"b3077576a91d36027f18f062e0a1ea252ea562ddfaf8d4f820b185de21f6c839"} Jul 08 18:13:03.263464 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.263115 2580 scope.go:117] "RemoveContainer" containerID="decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900" Jul 08 18:13:03.271676 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.271659 2580 scope.go:117] "RemoveContainer" containerID="decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900" Jul 08 18:13:03.271938 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:13:03.271911 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900\": container with ID starting with decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900 not found: ID does not exist" containerID="decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900" Jul 08 18:13:03.271990 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.271941 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900"} err="failed to get container status \"decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900\": rpc error: code = NotFound desc = could not find container \"decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900\": container with ID starting with decad7a28ed9a9b8bfce7bd966085f1cb8b7732679cad91f2786f2ddb7dbb900 not found: ID does not exist" Jul 08 18:13:03.282751 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.282728 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-82wb4"] Jul 08 18:13:03.287805 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:03.287776 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-82wb4"] Jul 08 18:13:04.869607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:04.869575 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c8c3723-97b2-4e42-a61d-436b50b71167" path="/var/lib/kubelet/pods/1c8c3723-97b2-4e42-a61d-436b50b71167/volumes" Jul 08 18:13:08.238411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:08.238384 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:13:08.446177 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:08.446146 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ds29x"] Jul 08 18:13:08.446358 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:08.446320 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" podUID="d194cf3a-b6a0-4467-9061-6506b1a54443" containerName="limitador" containerID="cri-o://54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e" gracePeriod=30 Jul 08 18:13:09.010883 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.010817 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:13:09.106141 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.106103 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d194cf3a-b6a0-4467-9061-6506b1a54443-config-file\") pod \"d194cf3a-b6a0-4467-9061-6506b1a54443\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " Jul 08 18:13:09.106329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.106210 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h96m7\" (UniqueName: \"kubernetes.io/projected/d194cf3a-b6a0-4467-9061-6506b1a54443-kube-api-access-h96m7\") pod \"d194cf3a-b6a0-4467-9061-6506b1a54443\" (UID: \"d194cf3a-b6a0-4467-9061-6506b1a54443\") " Jul 08 18:13:09.106510 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.106484 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d194cf3a-b6a0-4467-9061-6506b1a54443-config-file" (OuterVolumeSpecName: "config-file") pod "d194cf3a-b6a0-4467-9061-6506b1a54443" (UID: "d194cf3a-b6a0-4467-9061-6506b1a54443"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:13:09.108182 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.108153 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d194cf3a-b6a0-4467-9061-6506b1a54443-kube-api-access-h96m7" (OuterVolumeSpecName: "kube-api-access-h96m7") pod "d194cf3a-b6a0-4467-9061-6506b1a54443" (UID: "d194cf3a-b6a0-4467-9061-6506b1a54443"). InnerVolumeSpecName "kube-api-access-h96m7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:13:09.207040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.206992 2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d194cf3a-b6a0-4467-9061-6506b1a54443-config-file\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:13:09.207040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.207037 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h96m7\" (UniqueName: \"kubernetes.io/projected/d194cf3a-b6a0-4467-9061-6506b1a54443-kube-api-access-h96m7\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:13:09.294021 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.293942 2580 generic.go:356] "Generic (PLEG): container finished" podID="d194cf3a-b6a0-4467-9061-6506b1a54443" containerID="54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e" exitCode=0 Jul 08 18:13:09.294021 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.293989 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" Jul 08 18:13:09.294447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.294025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" event={"ID":"d194cf3a-b6a0-4467-9061-6506b1a54443","Type":"ContainerDied","Data":"54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e"} Jul 08 18:13:09.294447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.294078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ds29x" event={"ID":"d194cf3a-b6a0-4467-9061-6506b1a54443","Type":"ContainerDied","Data":"8bf120ce87d57202bc28739546627737b6550a3026ec7798f421703a3d47b9f0"} Jul 08 18:13:09.294447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.294101 2580 scope.go:117] "RemoveContainer" containerID="54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e" Jul 08 18:13:09.303179 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.303157 2580 scope.go:117] "RemoveContainer" containerID="54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e" Jul 08 18:13:09.303433 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:13:09.303417 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e\": container with ID starting with 54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e not found: ID does not exist" containerID="54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e" Jul 08 18:13:09.303482 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.303442 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e"} err="failed to get container status \"54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e\": rpc error: code = NotFound desc = could not find container \"54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e\": container with ID starting with 54f51455f002afcfbd95dcfff59b1d7fbc8a14034c0972dccdb8fb29b9a9f50e not found: ID does not exist" Jul 08 18:13:09.317891 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.317867 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ds29x"] Jul 08 18:13:09.324505 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:09.324485 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ds29x"] Jul 08 18:13:10.869277 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:10.869247 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d194cf3a-b6a0-4467-9061-6506b1a54443" path="/var/lib/kubelet/pods/d194cf3a-b6a0-4467-9061-6506b1a54443/volumes" Jul 08 18:13:11.750653 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:11.750583 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:12.016877 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.016764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:12.257672 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.257638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:12.298266 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.298177 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw"] Jul 08 18:13:12.298606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.298589 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c8c3723-97b2-4e42-a61d-436b50b71167" containerName="authorino" Jul 08 18:13:12.298692 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.298609 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c8c3723-97b2-4e42-a61d-436b50b71167" containerName="authorino" Jul 08 18:13:12.298692 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.298626 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d194cf3a-b6a0-4467-9061-6506b1a54443" containerName="limitador" Jul 08 18:13:12.298692 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.298635 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d194cf3a-b6a0-4467-9061-6506b1a54443" containerName="limitador" Jul 08 18:13:12.298868 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.298751 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d194cf3a-b6a0-4467-9061-6506b1a54443" containerName="limitador" Jul 08 18:13:12.298868 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.298767 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c8c3723-97b2-4e42-a61d-436b50b71167" containerName="authorino" Jul 08 18:13:12.303181 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.303161 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.307226 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.307202 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:13:12.307721 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.307705 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:13:12.308778 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.308722 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 08 18:13:12.311451 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.311432 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hl4xx\"" Jul 08 18:13:12.337162 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.337124 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw"] Jul 08 18:13:12.434094 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.434062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v2q7\" (UniqueName: \"kubernetes.io/projected/5177c403-3272-48ed-98e8-d3c809246c03-kube-api-access-4v2q7\") pod \"postgres-69fb4594cf-bhxxw\" (UID: \"5177c403-3272-48ed-98e8-d3c809246c03\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.434265 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.434136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5177c403-3272-48ed-98e8-d3c809246c03-data\") pod \"postgres-69fb4594cf-bhxxw\" (UID: \"5177c403-3272-48ed-98e8-d3c809246c03\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.535377 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.535327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5177c403-3272-48ed-98e8-d3c809246c03-data\") pod \"postgres-69fb4594cf-bhxxw\" (UID: \"5177c403-3272-48ed-98e8-d3c809246c03\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.535547 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.535478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v2q7\" (UniqueName: \"kubernetes.io/projected/5177c403-3272-48ed-98e8-d3c809246c03-kube-api-access-4v2q7\") pod \"postgres-69fb4594cf-bhxxw\" (UID: \"5177c403-3272-48ed-98e8-d3c809246c03\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.535750 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.535731 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5177c403-3272-48ed-98e8-d3c809246c03-data\") pod \"postgres-69fb4594cf-bhxxw\" (UID: \"5177c403-3272-48ed-98e8-d3c809246c03\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.545531 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.545505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v2q7\" (UniqueName: \"kubernetes.io/projected/5177c403-3272-48ed-98e8-d3c809246c03-kube-api-access-4v2q7\") pod \"postgres-69fb4594cf-bhxxw\" (UID: \"5177c403-3272-48ed-98e8-d3c809246c03\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.613565 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.613482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:12.763074 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:12.763051 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw"] Jul 08 18:13:12.764985 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:13:12.764953 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5177c403_3272_48ed_98e8_d3c809246c03.slice/crio-76ae9806a0cef994f98554c81c9be8c40b76f09aeaa58b6995f50976298511dc WatchSource:0}: Error finding container 76ae9806a0cef994f98554c81c9be8c40b76f09aeaa58b6995f50976298511dc: Status 404 returned error can't find the container with id 76ae9806a0cef994f98554c81c9be8c40b76f09aeaa58b6995f50976298511dc Jul 08 18:13:13.320113 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:13.320084 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" event={"ID":"5177c403-3272-48ed-98e8-d3c809246c03","Type":"ContainerStarted","Data":"76ae9806a0cef994f98554c81c9be8c40b76f09aeaa58b6995f50976298511dc"} Jul 08 18:13:18.348713 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:18.348677 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" event={"ID":"5177c403-3272-48ed-98e8-d3c809246c03","Type":"ContainerStarted","Data":"39fd2796e923f7c0a2017d4461ff304fc6b6bdde89df6427a34d01e18ff6c8f7"} Jul 08 18:13:18.349167 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:18.348775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:18.373260 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:18.373207 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" podStartSLOduration=1.684714745 podStartE2EDuration="6.373186768s" totalImagesPullingTime="4.688472023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:13:18.371089586 +0000 UTC m=+554.056296355" watchObservedRunningTime="2026-07-08 18:13:18.373186768 +0000 UTC m=+554.058393537" Jul 08 18:13:24.383382 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:24.383355 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bhxxw" Jul 08 18:13:24.763241 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:24.763206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:25.023143 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.023071 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j6nn5"] Jul 08 18:13:25.029641 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.029621 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" Jul 08 18:13:25.032864 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.032821 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7cf7r\"" Jul 08 18:13:25.039584 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.039564 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j6nn5"] Jul 08 18:13:25.152401 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.152365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgbn5\" (UniqueName: \"kubernetes.io/projected/bd59b3df-579f-44b7-8624-76ae50bbeea7-kube-api-access-xgbn5\") pod \"authorino-5d4b67c446-j6nn5\" (UID: \"bd59b3df-579f-44b7-8624-76ae50bbeea7\") " pod="kuadrant-system/authorino-5d4b67c446-j6nn5" Jul 08 18:13:25.253632 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.253598 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgbn5\" (UniqueName: \"kubernetes.io/projected/bd59b3df-579f-44b7-8624-76ae50bbeea7-kube-api-access-xgbn5\") pod \"authorino-5d4b67c446-j6nn5\" (UID: \"bd59b3df-579f-44b7-8624-76ae50bbeea7\") " pod="kuadrant-system/authorino-5d4b67c446-j6nn5" Jul 08 18:13:25.266210 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.266178 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgbn5\" (UniqueName: \"kubernetes.io/projected/bd59b3df-579f-44b7-8624-76ae50bbeea7-kube-api-access-xgbn5\") pod \"authorino-5d4b67c446-j6nn5\" (UID: \"bd59b3df-579f-44b7-8624-76ae50bbeea7\") " pod="kuadrant-system/authorino-5d4b67c446-j6nn5" Jul 08 18:13:25.274516 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.274455 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j6nn5"] Jul 08 18:13:25.274758 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.274745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" Jul 08 18:13:25.303466 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.303442 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7ff66c559b-mhlj5"] Jul 08 18:13:25.308988 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.308966 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" Jul 08 18:13:25.317884 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.317856 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ff66c559b-mhlj5"] Jul 08 18:13:25.423229 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.423205 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j6nn5"] Jul 08 18:13:25.425345 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:13:25.425321 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd59b3df_579f_44b7_8624_76ae50bbeea7.slice/crio-4b459f7fc386d39d718f50e9d849316b226669aa974394eac57f9d203801d9f7 WatchSource:0}: Error finding container 4b459f7fc386d39d718f50e9d849316b226669aa974394eac57f9d203801d9f7: Status 404 returned error can't find the container with id 4b459f7fc386d39d718f50e9d849316b226669aa974394eac57f9d203801d9f7 Jul 08 18:13:25.455541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.455512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49xn\" (UniqueName: \"kubernetes.io/projected/c617638e-9e03-4cbc-94ae-371b3fbf343b-kube-api-access-x49xn\") pod \"authorino-7ff66c559b-mhlj5\" (UID: \"c617638e-9e03-4cbc-94ae-371b3fbf343b\") " pod="kuadrant-system/authorino-7ff66c559b-mhlj5" Jul 08 18:13:25.470332 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.470281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:25.556354 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.556254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x49xn\" (UniqueName: \"kubernetes.io/projected/c617638e-9e03-4cbc-94ae-371b3fbf343b-kube-api-access-x49xn\") pod \"authorino-7ff66c559b-mhlj5\" (UID: \"c617638e-9e03-4cbc-94ae-371b3fbf343b\") " pod="kuadrant-system/authorino-7ff66c559b-mhlj5" Jul 08 18:13:25.566334 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.566302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49xn\" (UniqueName: \"kubernetes.io/projected/c617638e-9e03-4cbc-94ae-371b3fbf343b-kube-api-access-x49xn\") pod \"authorino-7ff66c559b-mhlj5\" (UID: \"c617638e-9e03-4cbc-94ae-371b3fbf343b\") " pod="kuadrant-system/authorino-7ff66c559b-mhlj5" Jul 08 18:13:25.621715 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.621684 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" Jul 08 18:13:25.753717 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.753689 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7ff66c559b-mhlj5"] Jul 08 18:13:25.754749 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:13:25.754725 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc617638e_9e03_4cbc_94ae_371b3fbf343b.slice/crio-cf14b5a81e9918dfdbe48d8c99c8f55103a7a18bb6a1d2c26193179caf61577a WatchSource:0}: Error finding container cf14b5a81e9918dfdbe48d8c99c8f55103a7a18bb6a1d2c26193179caf61577a: Status 404 returned error can't find the container with id cf14b5a81e9918dfdbe48d8c99c8f55103a7a18bb6a1d2c26193179caf61577a Jul 08 18:13:25.913250 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.913219 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7ff66c559b-mhlj5"] Jul 08 18:13:25.940762 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.940733 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b9bf588bd-mldtl"] Jul 08 18:13:25.944252 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.944233 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:25.948158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.948138 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 18:13:25.962991 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:25.962957 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b9bf588bd-mldtl"] Jul 08 18:13:26.059927 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.059905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nwr8\" (UniqueName: \"kubernetes.io/projected/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-kube-api-access-7nwr8\") pod \"authorino-7b9bf588bd-mldtl\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:26.060046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.059969 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-tls-cert\") pod \"authorino-7b9bf588bd-mldtl\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:26.160643 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.160593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-tls-cert\") pod \"authorino-7b9bf588bd-mldtl\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:26.160810 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.160684 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nwr8\" (UniqueName: \"kubernetes.io/projected/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-kube-api-access-7nwr8\") pod \"authorino-7b9bf588bd-mldtl\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:26.163098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.163075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-tls-cert\") pod \"authorino-7b9bf588bd-mldtl\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:26.173114 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.173092 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nwr8\" (UniqueName: \"kubernetes.io/projected/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-kube-api-access-7nwr8\") pod \"authorino-7b9bf588bd-mldtl\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:26.256464 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.256436 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:13:26.392039 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.391717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" event={"ID":"c617638e-9e03-4cbc-94ae-371b3fbf343b","Type":"ContainerStarted","Data":"570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6"} Jul 08 18:13:26.392039 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.391737 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" podUID="c617638e-9e03-4cbc-94ae-371b3fbf343b" containerName="authorino" containerID="cri-o://570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6" gracePeriod=30 Jul 08 18:13:26.392039 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.391766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" event={"ID":"c617638e-9e03-4cbc-94ae-371b3fbf343b","Type":"ContainerStarted","Data":"cf14b5a81e9918dfdbe48d8c99c8f55103a7a18bb6a1d2c26193179caf61577a"} Jul 08 18:13:26.392716 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.392692 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b9bf588bd-mldtl"] Jul 08 18:13:26.393459 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.393438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" event={"ID":"bd59b3df-579f-44b7-8624-76ae50bbeea7","Type":"ContainerStarted","Data":"9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777"} Jul 08 18:13:26.393553 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.393470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" event={"ID":"bd59b3df-579f-44b7-8624-76ae50bbeea7","Type":"ContainerStarted","Data":"4b459f7fc386d39d718f50e9d849316b226669aa974394eac57f9d203801d9f7"} Jul 08 18:13:26.393553 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.393527 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" podUID="bd59b3df-579f-44b7-8624-76ae50bbeea7" containerName="authorino" containerID="cri-o://9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777" gracePeriod=30 Jul 08 18:13:26.395128 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:13:26.395093 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5aed24f5_8587_477a_a7ec_3a0de5eea6d4.slice/crio-1901de95f7ead9e78293d84df0803ea3e6cf9f2857b423d43f2b7bc55a11ff92 WatchSource:0}: Error finding container 1901de95f7ead9e78293d84df0803ea3e6cf9f2857b423d43f2b7bc55a11ff92: Status 404 returned error can't find the container with id 1901de95f7ead9e78293d84df0803ea3e6cf9f2857b423d43f2b7bc55a11ff92 Jul 08 18:13:26.412064 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.412014 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" podStartSLOduration=1.137254021 podStartE2EDuration="1.41199485s" totalImagesPullingTime="274.740829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:13:26.408438541 +0000 UTC m=+562.093645427" watchObservedRunningTime="2026-07-08 18:13:26.41199485 +0000 UTC m=+562.097201620" Jul 08 18:13:26.434520 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.434473 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" podStartSLOduration=2.025104838 podStartE2EDuration="2.434457169s" totalImagesPullingTime="409.352331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:13:26.43297733 +0000 UTC m=+562.118184100" watchObservedRunningTime="2026-07-08 18:13:26.434457169 +0000 UTC m=+562.119663937" Jul 08 18:13:26.671391 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.671355 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" Jul 08 18:13:26.675145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.675115 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" Jul 08 18:13:26.767963 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.767940 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x49xn\" (UniqueName: \"kubernetes.io/projected/c617638e-9e03-4cbc-94ae-371b3fbf343b-kube-api-access-x49xn\") pod \"c617638e-9e03-4cbc-94ae-371b3fbf343b\" (UID: \"c617638e-9e03-4cbc-94ae-371b3fbf343b\") " Jul 08 18:13:26.768075 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.768051 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgbn5\" (UniqueName: \"kubernetes.io/projected/bd59b3df-579f-44b7-8624-76ae50bbeea7-kube-api-access-xgbn5\") pod \"bd59b3df-579f-44b7-8624-76ae50bbeea7\" (UID: \"bd59b3df-579f-44b7-8624-76ae50bbeea7\") " Jul 08 18:13:26.769907 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.769879 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c617638e-9e03-4cbc-94ae-371b3fbf343b-kube-api-access-x49xn" (OuterVolumeSpecName: "kube-api-access-x49xn") pod "c617638e-9e03-4cbc-94ae-371b3fbf343b" (UID: "c617638e-9e03-4cbc-94ae-371b3fbf343b"). InnerVolumeSpecName "kube-api-access-x49xn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:13:26.770030 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.770008 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd59b3df-579f-44b7-8624-76ae50bbeea7-kube-api-access-xgbn5" (OuterVolumeSpecName: "kube-api-access-xgbn5") pod "bd59b3df-579f-44b7-8624-76ae50bbeea7" (UID: "bd59b3df-579f-44b7-8624-76ae50bbeea7"). InnerVolumeSpecName "kube-api-access-xgbn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:13:26.869460 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.869377 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgbn5\" (UniqueName: \"kubernetes.io/projected/bd59b3df-579f-44b7-8624-76ae50bbeea7-kube-api-access-xgbn5\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:13:26.869460 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:26.869402 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x49xn\" (UniqueName: \"kubernetes.io/projected/c617638e-9e03-4cbc-94ae-371b3fbf343b-kube-api-access-x49xn\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:13:27.399518 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.399483 2580 generic.go:356] "Generic (PLEG): container finished" podID="c617638e-9e03-4cbc-94ae-371b3fbf343b" containerID="570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6" exitCode=0 Jul 08 18:13:27.399718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.399525 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" Jul 08 18:13:27.399718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.399574 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" event={"ID":"c617638e-9e03-4cbc-94ae-371b3fbf343b","Type":"ContainerDied","Data":"570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6"} Jul 08 18:13:27.399718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.399620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7ff66c559b-mhlj5" event={"ID":"c617638e-9e03-4cbc-94ae-371b3fbf343b","Type":"ContainerDied","Data":"cf14b5a81e9918dfdbe48d8c99c8f55103a7a18bb6a1d2c26193179caf61577a"} Jul 08 18:13:27.399718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.399642 2580 scope.go:117] "RemoveContainer" containerID="570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6" Jul 08 18:13:27.400935 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.400915 2580 generic.go:356] "Generic (PLEG): container finished" podID="bd59b3df-579f-44b7-8624-76ae50bbeea7" containerID="9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777" exitCode=0 Jul 08 18:13:27.401018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.400960 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" Jul 08 18:13:27.401018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.401002 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" event={"ID":"bd59b3df-579f-44b7-8624-76ae50bbeea7","Type":"ContainerDied","Data":"9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777"} Jul 08 18:13:27.401139 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.401045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j6nn5" event={"ID":"bd59b3df-579f-44b7-8624-76ae50bbeea7","Type":"ContainerDied","Data":"4b459f7fc386d39d718f50e9d849316b226669aa974394eac57f9d203801d9f7"} Jul 08 18:13:27.402619 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.402598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" event={"ID":"5aed24f5-8587-477a-a7ec-3a0de5eea6d4","Type":"ContainerStarted","Data":"092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d"} Jul 08 18:13:27.402719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.402629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" event={"ID":"5aed24f5-8587-477a-a7ec-3a0de5eea6d4","Type":"ContainerStarted","Data":"1901de95f7ead9e78293d84df0803ea3e6cf9f2857b423d43f2b7bc55a11ff92"} Jul 08 18:13:27.408690 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.408672 2580 scope.go:117] "RemoveContainer" containerID="570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6" Jul 08 18:13:27.408959 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:13:27.408941 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6\": container with ID starting with 570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6 not found: ID does not exist" containerID="570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6" Jul 08 18:13:27.409014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.408968 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6"} err="failed to get container status \"570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6\": rpc error: code = NotFound desc = could not find container \"570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6\": container with ID starting with 570499c0e49ec4f2a2b5d2c5f27e0b30f8f5f07ef3dc0dad07dde846b0a5d3f6 not found: ID does not exist" Jul 08 18:13:27.409014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.408986 2580 scope.go:117] "RemoveContainer" containerID="9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777" Jul 08 18:13:27.416666 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.416651 2580 scope.go:117] "RemoveContainer" containerID="9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777" Jul 08 18:13:27.416935 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:13:27.416908 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777\": container with ID starting with 9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777 not found: ID does not exist" containerID="9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777" Jul 08 18:13:27.417019 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.416940 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777"} err="failed to get container status \"9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777\": rpc error: code = NotFound desc = could not find container \"9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777\": container with ID starting with 9c2184f1dd9cdfa433b55d1323be3320a9ecadb1c47e24b33e79bf0f8d9fe777 not found: ID does not exist" Jul 08 18:13:27.428121 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.428075 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" podStartSLOduration=2.074830233 podStartE2EDuration="2.428060994s" totalImagesPullingTime="353.230761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:13:27.425912351 +0000 UTC m=+563.111119121" watchObservedRunningTime="2026-07-08 18:13:27.428060994 +0000 UTC m=+563.113267763" Jul 08 18:13:27.444005 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.443973 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7ff66c559b-mhlj5"] Jul 08 18:13:27.454018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.453993 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7ff66c559b-mhlj5"] Jul 08 18:13:27.475040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.475002 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j6nn5"] Jul 08 18:13:27.480340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:27.480312 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j6nn5"] Jul 08 18:13:28.869435 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:28.869407 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd59b3df-579f-44b7-8624-76ae50bbeea7" path="/var/lib/kubelet/pods/bd59b3df-579f-44b7-8624-76ae50bbeea7/volumes" Jul 08 18:13:28.869797 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:28.869787 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c617638e-9e03-4cbc-94ae-371b3fbf343b" path="/var/lib/kubelet/pods/c617638e-9e03-4cbc-94ae-371b3fbf343b/volumes" Jul 08 18:13:32.294172 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.294135 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:32.421815 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.421775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:32.510608 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.510578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:32.516203 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.516175 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-598cd5774c-mqbh6"] Jul 08 18:13:32.516746 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.516732 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c617638e-9e03-4cbc-94ae-371b3fbf343b" containerName="authorino" Jul 08 18:13:32.516793 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.516749 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c617638e-9e03-4cbc-94ae-371b3fbf343b" containerName="authorino" Jul 08 18:13:32.516793 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.516763 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd59b3df-579f-44b7-8624-76ae50bbeea7" containerName="authorino" Jul 08 18:13:32.516793 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.516769 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd59b3df-579f-44b7-8624-76ae50bbeea7" containerName="authorino" Jul 08 18:13:32.516918 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.516824 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd59b3df-579f-44b7-8624-76ae50bbeea7" containerName="authorino" Jul 08 18:13:32.516918 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.516847 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c617638e-9e03-4cbc-94ae-371b3fbf343b" containerName="authorino" Jul 08 18:13:32.630350 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.630257 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:32.663967 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.663930 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598cd5774c-mqbh6"] Jul 08 18:13:32.664120 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.664056 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:32.667752 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.667735 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 18:13:32.668548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.668532 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r4szr\"" Jul 08 18:13:32.668617 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.668564 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 18:13:32.822863 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.822804 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66365fb4-f954-405d-b8b9-63f417437b61-webhook-cert\") pod \"maas-controller-598cd5774c-mqbh6\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:32.823060 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.822960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-226v5\" (UniqueName: \"kubernetes.io/projected/66365fb4-f954-405d-b8b9-63f417437b61-kube-api-access-226v5\") pod \"maas-controller-598cd5774c-mqbh6\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:32.924085 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.924048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66365fb4-f954-405d-b8b9-63f417437b61-webhook-cert\") pod \"maas-controller-598cd5774c-mqbh6\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:32.924286 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.924151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-226v5\" (UniqueName: \"kubernetes.io/projected/66365fb4-f954-405d-b8b9-63f417437b61-kube-api-access-226v5\") pod \"maas-controller-598cd5774c-mqbh6\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:32.926469 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.926447 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66365fb4-f954-405d-b8b9-63f417437b61-webhook-cert\") pod \"maas-controller-598cd5774c-mqbh6\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:32.933351 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.933327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-226v5\" (UniqueName: \"kubernetes.io/projected/66365fb4-f954-405d-b8b9-63f417437b61-kube-api-access-226v5\") pod \"maas-controller-598cd5774c-mqbh6\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:32.974501 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:32.974471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:33.129181 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:33.129158 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598cd5774c-mqbh6"] Jul 08 18:13:33.131113 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:13:33.131084 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66365fb4_f954_405d_b8b9_63f417437b61.slice/crio-de0e7c160f9f45cd26781721b76deaead380dfaec46b5aaea7edcd3338bcf906 WatchSource:0}: Error finding container de0e7c160f9f45cd26781721b76deaead380dfaec46b5aaea7edcd3338bcf906: Status 404 returned error can't find the container with id de0e7c160f9f45cd26781721b76deaead380dfaec46b5aaea7edcd3338bcf906 Jul 08 18:13:33.434826 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:33.434792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598cd5774c-mqbh6" event={"ID":"66365fb4-f954-405d-b8b9-63f417437b61","Type":"ContainerStarted","Data":"de0e7c160f9f45cd26781721b76deaead380dfaec46b5aaea7edcd3338bcf906"} Jul 08 18:13:34.266228 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:34.266138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:35.876997 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:35.876949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:35.980253 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:35.980166 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:13:36.453408 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:36.453372 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598cd5774c-mqbh6" event={"ID":"66365fb4-f954-405d-b8b9-63f417437b61","Type":"ContainerStarted","Data":"cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b"} Jul 08 18:13:36.453592 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:36.453481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:13:36.483066 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:36.483009 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-598cd5774c-mqbh6" podStartSLOduration=1.926136918 podStartE2EDuration="4.482992307s" totalImagesPullingTime="2.556855389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:13:36.480728333 +0000 UTC m=+572.165935101" watchObservedRunningTime="2026-07-08 18:13:36.482992307 +0000 UTC m=+572.168199082" Jul 08 18:13:47.464653 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:13:47.464619 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:14:04.753812 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:04.753777 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:14:04.755105 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:04.755068 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:04.756413 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:04.756392 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:14:07.283006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:07.282876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:07.907162 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:07.907124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:08.019642 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.019609 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:08.539472 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.539435 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:08.682715 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.682675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:08.831914 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.831824 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-d8l8b"] Jul 08 18:14:08.835218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.835202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:08.857429 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.857410 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qphj5\"" Jul 08 18:14:08.876642 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.876619 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-d8l8b"] Jul 08 18:14:08.892986 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.892965 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 18:14:08.931166 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.931130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d010009-a12f-424c-ae94-96042a1fedd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-d8l8b\" (UID: \"5d010009-a12f-424c-ae94-96042a1fedd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:08.931317 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:08.931256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlcmr\" (UniqueName: \"kubernetes.io/projected/5d010009-a12f-424c-ae94-96042a1fedd2-kube-api-access-nlcmr\") pod \"payload-pre-processing-7964457696-d8l8b\" (UID: \"5d010009-a12f-424c-ae94-96042a1fedd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:09.032773 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:09.032717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d010009-a12f-424c-ae94-96042a1fedd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-d8l8b\" (UID: \"5d010009-a12f-424c-ae94-96042a1fedd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:09.032956 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:09.032897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlcmr\" (UniqueName: \"kubernetes.io/projected/5d010009-a12f-424c-ae94-96042a1fedd2-kube-api-access-nlcmr\") pod \"payload-pre-processing-7964457696-d8l8b\" (UID: \"5d010009-a12f-424c-ae94-96042a1fedd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:09.033518 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:09.033497 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d010009-a12f-424c-ae94-96042a1fedd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-d8l8b\" (UID: \"5d010009-a12f-424c-ae94-96042a1fedd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:09.052175 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:09.052146 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlcmr\" (UniqueName: \"kubernetes.io/projected/5d010009-a12f-424c-ae94-96042a1fedd2-kube-api-access-nlcmr\") pod \"payload-pre-processing-7964457696-d8l8b\" (UID: \"5d010009-a12f-424c-ae94-96042a1fedd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:09.145420 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:09.145391 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:09.316778 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:09.316747 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-d8l8b"] Jul 08 18:14:09.318736 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:14:09.318711 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d010009_a12f_424c_ae94_96042a1fedd2.slice/crio-eb178af2aaa922fdd71834aab50ea9cbac7ed4bb4b8c88020987a64ef3a3531c WatchSource:0}: Error finding container eb178af2aaa922fdd71834aab50ea9cbac7ed4bb4b8c88020987a64ef3a3531c: Status 404 returned error can't find the container with id eb178af2aaa922fdd71834aab50ea9cbac7ed4bb4b8c88020987a64ef3a3531c Jul 08 18:14:09.620549 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:09.620455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" event={"ID":"5d010009-a12f-424c-ae94-96042a1fedd2","Type":"ContainerStarted","Data":"eb178af2aaa922fdd71834aab50ea9cbac7ed4bb4b8c88020987a64ef3a3531c"} Jul 08 18:14:11.633549 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:11.633441 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" event={"ID":"5d010009-a12f-424c-ae94-96042a1fedd2","Type":"ContainerStarted","Data":"aebe1946eb6904855362b0d80a47b43202fbb21e34afc19ae487c5eb5eda4069"} Jul 08 18:14:11.634339 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:11.634313 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:11.656871 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:11.656804 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" podStartSLOduration=1.6781054659999999 podStartE2EDuration="3.656783833s" totalImagesPullingTime="1.978678367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:14:11.654930453 +0000 UTC m=+607.340137220" watchObservedRunningTime="2026-07-08 18:14:11.656783833 +0000 UTC m=+607.341990601" Jul 08 18:14:15.711410 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:15.711369 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:17.669993 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:17.669962 2580 generic.go:356] "Generic (PLEG): container finished" podID="59a21420-9066-4247-939c-aef787050364" containerID="16803c63eb99f0a01480050285f07c719fd96ae50c81fbc97d0de58545f65654" exitCode=0 Jul 08 18:14:17.670397 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:17.670034 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" event={"ID":"59a21420-9066-4247-939c-aef787050364","Type":"ContainerDied","Data":"16803c63eb99f0a01480050285f07c719fd96ae50c81fbc97d0de58545f65654"} Jul 08 18:14:17.670444 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:17.670410 2580 scope.go:117] "RemoveContainer" containerID="16803c63eb99f0a01480050285f07c719fd96ae50c81fbc97d0de58545f65654" Jul 08 18:14:17.671443 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:17.671412 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:14:17.671532 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:17.671498 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:14:17.671571 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:17.671555 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:14:18.636604 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:18.636567 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:18.677218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:18.677187 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" event={"ID":"59a21420-9066-4247-939c-aef787050364","Type":"ContainerStarted","Data":"f6437d87384265247bb75298a34b1305fed7cdb0c3440b178b8138b56ad9a9e9"} Jul 08 18:14:19.308266 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:19.308231 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:14:19.309282 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:19.309257 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vfz68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 18:14:19.309399 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:19.309304 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" podUID="59a21420-9066-4247-939c-aef787050364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 18:14:20.187213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:20.187182 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:14:20.307953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:20.307922 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vfz68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 18:14:20.308108 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:20.307972 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" podUID="59a21420-9066-4247-939c-aef787050364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 18:14:21.308116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:21.308083 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vfz68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 18:14:21.308481 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:21.308134 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" podUID="59a21420-9066-4247-939c-aef787050364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 18:14:22.307951 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:22.307906 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vfz68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 18:14:22.308141 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:22.307977 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" podUID="59a21420-9066-4247-939c-aef787050364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 18:14:23.312054 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:23.312027 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:14:23.313075 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:23.313053 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vfz68" Jul 08 18:14:23.649349 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:23.649324 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-d8l8b" Jul 08 18:14:26.652844 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.652801 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz"] Jul 08 18:14:26.656697 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.656678 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.662258 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.662232 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 18:14:26.662382 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.662362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-l6gpw\"" Jul 08 18:14:26.662456 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.662306 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 18:14:26.662543 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.662287 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 18:14:26.680331 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.680311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz"] Jul 08 18:14:26.704199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.704173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.704318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.704210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4166db4f-acea-4220-bc8f-0c8636758a5a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.704318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.704243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.704318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.704260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.704318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.704298 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.704533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.704345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjnpz\" (UniqueName: \"kubernetes.io/projected/4166db4f-acea-4220-bc8f-0c8636758a5a-kube-api-access-cjnpz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.805363 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.805547 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4166db4f-acea-4220-bc8f-0c8636758a5a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.805547 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.805547 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.805547 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.805547 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjnpz\" (UniqueName: \"kubernetes.io/projected/4166db4f-acea-4220-bc8f-0c8636758a5a-kube-api-access-cjnpz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.805890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805859 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.806008 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805888 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.806008 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.805919 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.807467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.807448 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4166db4f-acea-4220-bc8f-0c8636758a5a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.807909 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.807890 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4166db4f-acea-4220-bc8f-0c8636758a5a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.816438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.816417 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjnpz\" (UniqueName: \"kubernetes.io/projected/4166db4f-acea-4220-bc8f-0c8636758a5a-kube-api-access-cjnpz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz\" (UID: \"4166db4f-acea-4220-bc8f-0c8636758a5a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:26.966862 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:26.966775 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:27.113297 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:27.113271 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz"] Jul 08 18:14:27.114889 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:14:27.114856 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4166db4f_acea_4220_bc8f_0c8636758a5a.slice/crio-4fb8661854b2849152a00bf5fcb5e5ffcf9080e8ae4540c476cbdb76f4e2d5b7 WatchSource:0}: Error finding container 4fb8661854b2849152a00bf5fcb5e5ffcf9080e8ae4540c476cbdb76f4e2d5b7: Status 404 returned error can't find the container with id 4fb8661854b2849152a00bf5fcb5e5ffcf9080e8ae4540c476cbdb76f4e2d5b7 Jul 08 18:14:27.202219 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:27.202183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:27.724270 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:27.724229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" event={"ID":"4166db4f-acea-4220-bc8f-0c8636758a5a","Type":"ContainerStarted","Data":"4fb8661854b2849152a00bf5fcb5e5ffcf9080e8ae4540c476cbdb76f4e2d5b7"} Jul 08 18:14:28.730563 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:28.730526 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" event={"ID":"4166db4f-acea-4220-bc8f-0c8636758a5a","Type":"ContainerStarted","Data":"745238da93482fb04bc8dbb9fc2b4d60453c5893ab81756318dcd872bd71f422"} Jul 08 18:14:28.796015 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:28.795957 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" podStartSLOduration=1.628710781 podStartE2EDuration="2.79594334s" totalImagesPullingTime="1.167232559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:14:28.795445416 +0000 UTC m=+624.480652186" watchObservedRunningTime="2026-07-08 18:14:28.79594334 +0000 UTC m=+624.481150109" Jul 08 18:14:36.966941 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:36.966905 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:36.967593 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:36.966969 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:36.979907 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:36.979885 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:37.788138 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:37.788108 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz" Jul 08 18:14:41.425675 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:41.425644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:49.667001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:49.666959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:51.602982 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:51.602951 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:55.454941 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.454906 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2"] Jul 08 18:14:55.459511 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.459491 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.462437 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.462415 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 18:14:55.472378 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.472357 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2"] Jul 08 18:14:55.505115 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.502797 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:14:55.570518 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.570490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.570674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.570550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.570674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.570621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.570674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.570650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q49xr\" (UniqueName: \"kubernetes.io/projected/8a9a32d9-5490-4845-8750-f7ba11245d95-kube-api-access-q49xr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.570774 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.570681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.570774 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.570713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9a32d9-5490-4845-8750-f7ba11245d95-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.671710 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.671681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.671865 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.671738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.671865 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.671774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.671865 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.671791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q49xr\" (UniqueName: \"kubernetes.io/projected/8a9a32d9-5490-4845-8750-f7ba11245d95-kube-api-access-q49xr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.671865 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.671825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.672077 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.671983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9a32d9-5490-4845-8750-f7ba11245d95-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.672155 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.672134 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.672215 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.672171 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.672278 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.672212 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.674040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.674020 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9a32d9-5490-4845-8750-f7ba11245d95-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.674328 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.674311 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9a32d9-5490-4845-8750-f7ba11245d95-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.680509 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.680484 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q49xr\" (UniqueName: \"kubernetes.io/projected/8a9a32d9-5490-4845-8750-f7ba11245d95-kube-api-access-q49xr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2\" (UID: \"8a9a32d9-5490-4845-8750-f7ba11245d95\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.770398 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.770318 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:14:55.911878 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.911855 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2"] Jul 08 18:14:55.913812 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:14:55.913789 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9a32d9_5490_4845_8750_f7ba11245d95.slice/crio-f1528e173380c7b31252f5488547336dca1a4a81cde3f83cacc681729ba742ea WatchSource:0}: Error finding container f1528e173380c7b31252f5488547336dca1a4a81cde3f83cacc681729ba742ea: Status 404 returned error can't find the container with id f1528e173380c7b31252f5488547336dca1a4a81cde3f83cacc681729ba742ea Jul 08 18:14:55.915070 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:55.915054 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:14:56.872568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:56.872534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" event={"ID":"8a9a32d9-5490-4845-8750-f7ba11245d95","Type":"ContainerStarted","Data":"749c5f5b3da3017e38f12f5e6171c9992c804b43ef69834f087101f49cf2a8f0"} Jul 08 18:14:56.872994 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:56.872590 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" event={"ID":"8a9a32d9-5490-4845-8750-f7ba11245d95","Type":"ContainerStarted","Data":"f1528e173380c7b31252f5488547336dca1a4a81cde3f83cacc681729ba742ea"} Jul 08 18:14:56.900652 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:14:56.900603 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" podStartSLOduration=1.706626349 podStartE2EDuration="1.900587161s" totalImagesPullingTime="193.960812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:14:56.899053476 +0000 UTC m=+652.584260224" watchObservedRunningTime="2026-07-08 18:14:56.900587161 +0000 UTC m=+652.585793934" Jul 08 18:15:05.770962 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:15:05.770927 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:15:05.771354 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:15:05.770997 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:15:05.783494 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:15:05.783469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:15:05.932010 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:15:05.931983 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2" Jul 08 18:15:06.586920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:15:06.586890 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:15:11.161608 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:15:11.161567 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:15:14.235970 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:15:14.235933 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:16:21.044934 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.044864 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85dc8bd77-cwbgl"] Jul 08 18:16:21.048543 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.048526 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.057219 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.057195 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85dc8bd77-cwbgl"] Jul 08 18:16:21.125886 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.125856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f697f0a-416e-4cc8-baec-e7fe8d2e45ca-tls-cert\") pod \"authorino-85dc8bd77-cwbgl\" (UID: \"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca\") " pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.126039 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.125925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf49n\" (UniqueName: \"kubernetes.io/projected/9f697f0a-416e-4cc8-baec-e7fe8d2e45ca-kube-api-access-hf49n\") pod \"authorino-85dc8bd77-cwbgl\" (UID: \"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca\") " pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.226636 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.226611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f697f0a-416e-4cc8-baec-e7fe8d2e45ca-tls-cert\") pod \"authorino-85dc8bd77-cwbgl\" (UID: \"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca\") " pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.226811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.226660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf49n\" (UniqueName: \"kubernetes.io/projected/9f697f0a-416e-4cc8-baec-e7fe8d2e45ca-kube-api-access-hf49n\") pod \"authorino-85dc8bd77-cwbgl\" (UID: \"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca\") " pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.228945 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.228919 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9f697f0a-416e-4cc8-baec-e7fe8d2e45ca-tls-cert\") pod \"authorino-85dc8bd77-cwbgl\" (UID: \"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca\") " pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.235848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.235815 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf49n\" (UniqueName: \"kubernetes.io/projected/9f697f0a-416e-4cc8-baec-e7fe8d2e45ca-kube-api-access-hf49n\") pod \"authorino-85dc8bd77-cwbgl\" (UID: \"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca\") " pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.358631 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.358555 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85dc8bd77-cwbgl" Jul 08 18:16:21.499314 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:21.499290 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85dc8bd77-cwbgl"] Jul 08 18:16:21.500860 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:16:21.500821 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f697f0a_416e_4cc8_baec_e7fe8d2e45ca.slice/crio-1b6e162527c7e5d0c0f24ba468e709e6c9e36589d6f565bca32808db3129e056 WatchSource:0}: Error finding container 1b6e162527c7e5d0c0f24ba468e709e6c9e36589d6f565bca32808db3129e056: Status 404 returned error can't find the container with id 1b6e162527c7e5d0c0f24ba468e709e6c9e36589d6f565bca32808db3129e056 Jul 08 18:16:22.308619 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.308578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85dc8bd77-cwbgl" event={"ID":"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca","Type":"ContainerStarted","Data":"ce8ddbd7564c8ae179b81371547c50a1cea2d7566ebead4e770fb304dc7fc426"} Jul 08 18:16:22.309114 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.308634 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85dc8bd77-cwbgl" event={"ID":"9f697f0a-416e-4cc8-baec-e7fe8d2e45ca","Type":"ContainerStarted","Data":"1b6e162527c7e5d0c0f24ba468e709e6c9e36589d6f565bca32808db3129e056"} Jul 08 18:16:22.331285 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.331217 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85dc8bd77-cwbgl" podStartSLOduration=1.001869916 podStartE2EDuration="1.331194504s" totalImagesPullingTime="329.324588ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:16:22.327411984 +0000 UTC m=+738.012618754" watchObservedRunningTime="2026-07-08 18:16:22.331194504 +0000 UTC m=+738.016401274" Jul 08 18:16:22.380109 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.380069 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b9bf588bd-mldtl"] Jul 08 18:16:22.380292 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.380253 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" podUID="5aed24f5-8587-477a-a7ec-3a0de5eea6d4" containerName="authorino" containerID="cri-o://092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d" gracePeriod=30 Jul 08 18:16:22.582944 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.582917 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:16:22.654710 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.654676 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:16:22.678694 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.678656 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:16:22.740780 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.740745 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-tls-cert\") pod \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " Jul 08 18:16:22.740978 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.740793 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nwr8\" (UniqueName: \"kubernetes.io/projected/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-kube-api-access-7nwr8\") pod \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\" (UID: \"5aed24f5-8587-477a-a7ec-3a0de5eea6d4\") " Jul 08 18:16:22.743340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.743307 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-kube-api-access-7nwr8" (OuterVolumeSpecName: "kube-api-access-7nwr8") pod "5aed24f5-8587-477a-a7ec-3a0de5eea6d4" (UID: "5aed24f5-8587-477a-a7ec-3a0de5eea6d4"). InnerVolumeSpecName "kube-api-access-7nwr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:16:22.752411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.752376 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5aed24f5-8587-477a-a7ec-3a0de5eea6d4" (UID: "5aed24f5-8587-477a-a7ec-3a0de5eea6d4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:16:22.842442 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.842366 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-tls-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:16:22.842442 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:22.842395 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nwr8\" (UniqueName: \"kubernetes.io/projected/5aed24f5-8587-477a-a7ec-3a0de5eea6d4-kube-api-access-7nwr8\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:16:23.315424 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.315387 2580 generic.go:356] "Generic (PLEG): container finished" podID="5aed24f5-8587-477a-a7ec-3a0de5eea6d4" containerID="092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d" exitCode=0 Jul 08 18:16:23.315807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.315429 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" Jul 08 18:16:23.315807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.315471 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" event={"ID":"5aed24f5-8587-477a-a7ec-3a0de5eea6d4","Type":"ContainerDied","Data":"092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d"} Jul 08 18:16:23.315807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.315518 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b9bf588bd-mldtl" event={"ID":"5aed24f5-8587-477a-a7ec-3a0de5eea6d4","Type":"ContainerDied","Data":"1901de95f7ead9e78293d84df0803ea3e6cf9f2857b423d43f2b7bc55a11ff92"} Jul 08 18:16:23.315807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.315535 2580 scope.go:117] "RemoveContainer" containerID="092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d" Jul 08 18:16:23.325160 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.325137 2580 scope.go:117] "RemoveContainer" containerID="092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d" Jul 08 18:16:23.325491 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:16:23.325469 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d\": container with ID starting with 092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d not found: ID does not exist" containerID="092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d" Jul 08 18:16:23.325568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.325503 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d"} err="failed to get container status \"092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d\": rpc error: code = NotFound desc = could not find container \"092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d\": container with ID starting with 092fb2f161c3df09c9c01db546fd23d05870e0f61b15b2cd676cbe264b507f9d not found: ID does not exist" Jul 08 18:16:23.337936 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.337910 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b9bf588bd-mldtl"] Jul 08 18:16:23.342340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:23.342319 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b9bf588bd-mldtl"] Jul 08 18:16:24.869937 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:24.869905 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5aed24f5-8587-477a-a7ec-3a0de5eea6d4" path="/var/lib/kubelet/pods/5aed24f5-8587-477a-a7ec-3a0de5eea6d4/volumes" Jul 08 18:16:35.029274 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:16:35.029235 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:17:33.144404 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.144316 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:17:33.242369 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.242335 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-598cd5774c-mqbh6"] Jul 08 18:17:33.242655 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.242629 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-598cd5774c-mqbh6" podUID="66365fb4-f954-405d-b8b9-63f417437b61" containerName="manager" containerID="cri-o://cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b" gracePeriod=30 Jul 08 18:17:33.497182 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.497160 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:17:33.687990 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.687908 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66365fb4-f954-405d-b8b9-63f417437b61-webhook-cert\") pod \"66365fb4-f954-405d-b8b9-63f417437b61\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " Jul 08 18:17:33.687990 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.687941 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-226v5\" (UniqueName: \"kubernetes.io/projected/66365fb4-f954-405d-b8b9-63f417437b61-kube-api-access-226v5\") pod \"66365fb4-f954-405d-b8b9-63f417437b61\" (UID: \"66365fb4-f954-405d-b8b9-63f417437b61\") " Jul 08 18:17:33.690190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.690154 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66365fb4-f954-405d-b8b9-63f417437b61-kube-api-access-226v5" (OuterVolumeSpecName: "kube-api-access-226v5") pod "66365fb4-f954-405d-b8b9-63f417437b61" (UID: "66365fb4-f954-405d-b8b9-63f417437b61"). InnerVolumeSpecName "kube-api-access-226v5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:17:33.690332 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.690313 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66365fb4-f954-405d-b8b9-63f417437b61-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "66365fb4-f954-405d-b8b9-63f417437b61" (UID: "66365fb4-f954-405d-b8b9-63f417437b61"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:17:33.692674 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.692644 2580 generic.go:356] "Generic (PLEG): container finished" podID="66365fb4-f954-405d-b8b9-63f417437b61" containerID="cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b" exitCode=0 Jul 08 18:17:33.692782 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.692696 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598cd5774c-mqbh6" Jul 08 18:17:33.692782 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.692724 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598cd5774c-mqbh6" event={"ID":"66365fb4-f954-405d-b8b9-63f417437b61","Type":"ContainerDied","Data":"cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b"} Jul 08 18:17:33.692877 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.692796 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598cd5774c-mqbh6" event={"ID":"66365fb4-f954-405d-b8b9-63f417437b61","Type":"ContainerDied","Data":"de0e7c160f9f45cd26781721b76deaead380dfaec46b5aaea7edcd3338bcf906"} Jul 08 18:17:33.692877 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.692818 2580 scope.go:117] "RemoveContainer" containerID="cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b" Jul 08 18:17:33.709428 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.709412 2580 scope.go:117] "RemoveContainer" containerID="cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b" Jul 08 18:17:33.709700 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:17:33.709683 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b\": container with ID starting with cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b not found: ID does not exist" containerID="cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b" Jul 08 18:17:33.709751 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.709710 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b"} err="failed to get container status \"cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b\": rpc error: code = NotFound desc = could not find container \"cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b\": container with ID starting with cbb18fa510752071a0dc8f07def75a2a636f0fd40e4dc3eb6d219a32fc85ed8b not found: ID does not exist" Jul 08 18:17:33.722196 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.722171 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-598cd5774c-mqbh6"] Jul 08 18:17:33.729818 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.729793 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-598cd5774c-mqbh6"] Jul 08 18:17:33.788922 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.788896 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66365fb4-f954-405d-b8b9-63f417437b61-webhook-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:17:33.788922 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:33.788919 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-226v5\" (UniqueName: \"kubernetes.io/projected/66365fb4-f954-405d-b8b9-63f417437b61-kube-api-access-226v5\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:17:34.400426 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.400391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:17:34.511950 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.511911 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-598cd5774c-ssxx4"] Jul 08 18:17:34.512383 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.512365 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66365fb4-f954-405d-b8b9-63f417437b61" containerName="manager" Jul 08 18:17:34.512478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.512386 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66365fb4-f954-405d-b8b9-63f417437b61" containerName="manager" Jul 08 18:17:34.512478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.512402 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aed24f5-8587-477a-a7ec-3a0de5eea6d4" containerName="authorino" Jul 08 18:17:34.512478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.512409 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aed24f5-8587-477a-a7ec-3a0de5eea6d4" containerName="authorino" Jul 08 18:17:34.512630 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.512523 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66365fb4-f954-405d-b8b9-63f417437b61" containerName="manager" Jul 08 18:17:34.512630 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.512542 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aed24f5-8587-477a-a7ec-3a0de5eea6d4" containerName="authorino" Jul 08 18:17:34.517088 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.517066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.519820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.519795 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r4szr\"" Jul 08 18:17:34.520931 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.520906 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 18:17:34.521061 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.520955 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 18:17:34.527736 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.527712 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598cd5774c-ssxx4"] Jul 08 18:17:34.697744 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.697664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33add953-1c37-468e-bd2f-9c112d51bcb1-webhook-cert\") pod \"maas-controller-598cd5774c-ssxx4\" (UID: \"33add953-1c37-468e-bd2f-9c112d51bcb1\") " pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.697924 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.697809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmrdz\" (UniqueName: \"kubernetes.io/projected/33add953-1c37-468e-bd2f-9c112d51bcb1-kube-api-access-cmrdz\") pod \"maas-controller-598cd5774c-ssxx4\" (UID: \"33add953-1c37-468e-bd2f-9c112d51bcb1\") " pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.798749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.798714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33add953-1c37-468e-bd2f-9c112d51bcb1-webhook-cert\") pod \"maas-controller-598cd5774c-ssxx4\" (UID: \"33add953-1c37-468e-bd2f-9c112d51bcb1\") " pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.798941 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.798798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmrdz\" (UniqueName: \"kubernetes.io/projected/33add953-1c37-468e-bd2f-9c112d51bcb1-kube-api-access-cmrdz\") pod \"maas-controller-598cd5774c-ssxx4\" (UID: \"33add953-1c37-468e-bd2f-9c112d51bcb1\") " pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.801155 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.801125 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33add953-1c37-468e-bd2f-9c112d51bcb1-webhook-cert\") pod \"maas-controller-598cd5774c-ssxx4\" (UID: \"33add953-1c37-468e-bd2f-9c112d51bcb1\") " pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.807848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.807814 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmrdz\" (UniqueName: \"kubernetes.io/projected/33add953-1c37-468e-bd2f-9c112d51bcb1-kube-api-access-cmrdz\") pod \"maas-controller-598cd5774c-ssxx4\" (UID: \"33add953-1c37-468e-bd2f-9c112d51bcb1\") " pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.828106 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.828081 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:34.870947 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.870909 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66365fb4-f954-405d-b8b9-63f417437b61" path="/var/lib/kubelet/pods/66365fb4-f954-405d-b8b9-63f417437b61/volumes" Jul 08 18:17:34.963397 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:34.963369 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-598cd5774c-ssxx4"] Jul 08 18:17:34.965087 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:17:34.965059 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33add953_1c37_468e_bd2f_9c112d51bcb1.slice/crio-e216938fc33b60c517c6766541d0d32333c7fde088bc1558f56b0a1b14ce1b3c WatchSource:0}: Error finding container e216938fc33b60c517c6766541d0d32333c7fde088bc1558f56b0a1b14ce1b3c: Status 404 returned error can't find the container with id e216938fc33b60c517c6766541d0d32333c7fde088bc1558f56b0a1b14ce1b3c Jul 08 18:17:35.705075 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:35.705042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598cd5774c-ssxx4" event={"ID":"33add953-1c37-468e-bd2f-9c112d51bcb1","Type":"ContainerStarted","Data":"2bbc21953d7e50633cd3702f09397b7b9b8d6747332cd4b549ac8fddab420f72"} Jul 08 18:17:35.705452 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:35.705106 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:17:35.705452 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:35.705129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-598cd5774c-ssxx4" event={"ID":"33add953-1c37-468e-bd2f-9c112d51bcb1","Type":"ContainerStarted","Data":"e216938fc33b60c517c6766541d0d32333c7fde088bc1558f56b0a1b14ce1b3c"} Jul 08 18:17:35.723006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:35.722965 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-598cd5774c-ssxx4" podStartSLOduration=1.163896805 podStartE2EDuration="1.722949783s" totalImagesPullingTime="559.052978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:17:35.722781298 +0000 UTC m=+811.407988070" watchObservedRunningTime="2026-07-08 18:17:35.722949783 +0000 UTC m=+811.408156554" Jul 08 18:17:46.716635 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:17:46.716605 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-598cd5774c-ssxx4" Jul 08 18:18:15.450272 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:15.450233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:15.548138 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:15.548087 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:15.669965 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:15.669925 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:21.699196 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:21.699149 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:21.815750 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:21.815716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:21.900187 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:21.900158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:22.095718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:22.095603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:23.673238 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:23.673202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:28.275595 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:28.275557 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:18:28.336679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:18:28.336646 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:19:04.786297 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:19:04.786213 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:19:04.787703 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:19:04.787674 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:20:37.619332 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:20:37.619255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:20:39.524030 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:20:39.523991 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:21:19.919159 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:21:19.919119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:21:30.193854 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:21:30.193796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:22:13.301325 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:22:13.301212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:24:04.816723 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:24:04.816693 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:24:04.818902 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:24:04.818878 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:27:30.830086 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:27:30.830055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:27:32.433496 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:27:32.433460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:28:40.429245 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:28:40.429212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:04.845894 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:04.845864 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:29:04.850137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:04.850117 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:29:06.641108 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:06.641066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:12.058170 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:12.058120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:20.304682 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:20.304625 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:25.433883 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:25.433814 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:28.571995 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:28.571903 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:28.669756 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:28.669723 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:29.055554 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:29.055516 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:35.406539 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.406504 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:35.467740 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.467709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:35.603873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.603828 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj"] Jul 08 18:29:35.608104 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.608087 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.610910 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.610888 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ef85937c-openshift-default-dockercfg-znpsr\"" Jul 08 18:29:35.629240 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8djg\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-kube-api-access-q8djg\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629336 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-envoy\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629336 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-token\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629336 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-socket\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629336 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-data\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629489 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-certs\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629489 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istiod-ca-cert\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629489 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-credential-socket\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.629489 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.629479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-podinfo\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.650185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.650161 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:35.651807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.651785 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj"] Jul 08 18:29:35.730685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-certs\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.730685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istiod-ca-cert\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.730685 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-credential-socket\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730699 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-podinfo\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8djg\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-kube-api-access-q8djg\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-envoy\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-token\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-socket\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.730862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-data\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.731080 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-certs\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.731205 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-credential-socket\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.731376 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-socket\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731486 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.731447 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istiod-ca-cert\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.731486 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.731448 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-data\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.733069 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.733044 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-envoy\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.733304 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.733286 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-podinfo\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.756921 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.756895 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-token\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.757359 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.757344 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8djg\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-kube-api-access-q8djg\") pod \"e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:35.759356 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.759332 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:35.918969 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:35.918939 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:36.069634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.069611 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj"] Jul 08 18:29:36.072231 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:29:36.072201 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e16909_9f6f_4bfa_802a_6ac7657be3ab.slice/crio-2c099aa04a66d199be384ed0e961876e08cdcfc90a84d759a6998dfe24186edb WatchSource:0}: Error finding container 2c099aa04a66d199be384ed0e961876e08cdcfc90a84d759a6998dfe24186edb: Status 404 returned error can't find the container with id 2c099aa04a66d199be384ed0e961876e08cdcfc90a84d759a6998dfe24186edb Jul 08 18:29:36.074315 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.074282 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:29:36.074400 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.074348 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:29:36.074400 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.074375 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:29:36.325623 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.325540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" event={"ID":"f9e16909-9f6f-4bfa-802a-6ac7657be3ab","Type":"ContainerStarted","Data":"206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1"} Jul 08 18:29:36.325623 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.325601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" event={"ID":"f9e16909-9f6f-4bfa-802a-6ac7657be3ab","Type":"ContainerStarted","Data":"2c099aa04a66d199be384ed0e961876e08cdcfc90a84d759a6998dfe24186edb"} Jul 08 18:29:36.350906 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.350859 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" podStartSLOduration=1.35082754 podStartE2EDuration="1.35082754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:29:36.349551937 +0000 UTC m=+1532.034758707" watchObservedRunningTime="2026-07-08 18:29:36.35082754 +0000 UTC m=+1532.036034308" Jul 08 18:29:36.751203 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.751165 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:36.850252 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.850209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:36.920100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:36.920073 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:37.029062 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:37.028968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:37.144019 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:37.143955 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:37.456295 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:37.456247 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:37.523521 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:37.523490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:37.924319 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:37.924290 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:38.339629 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:38.339547 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:38.340399 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:38.340378 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:39.674993 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:39.674957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:39.913393 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:39.913359 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:40.107211 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:40.107173 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:40.208259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:40.208213 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:41.919211 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:41.919179 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:42.082557 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:42.082521 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj"] Jul 08 18:29:42.082778 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:42.082706 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" podUID="f9e16909-9f6f-4bfa-802a-6ac7657be3ab" containerName="istio-proxy" containerID="cri-o://206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1" gracePeriod=30 Jul 08 18:29:42.125297 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:42.125261 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:42.206974 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:42.206889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:42.373375 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:42.373347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:43.693661 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:43.693592 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:47.335073 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.335051 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:47.388122 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.388092 2580 generic.go:356] "Generic (PLEG): container finished" podID="f9e16909-9f6f-4bfa-802a-6ac7657be3ab" containerID="206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1" exitCode=0 Jul 08 18:29:47.388261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.388141 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" Jul 08 18:29:47.388261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.388157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" event={"ID":"f9e16909-9f6f-4bfa-802a-6ac7657be3ab","Type":"ContainerDied","Data":"206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1"} Jul 08 18:29:47.388261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.388202 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj" event={"ID":"f9e16909-9f6f-4bfa-802a-6ac7657be3ab","Type":"ContainerDied","Data":"2c099aa04a66d199be384ed0e961876e08cdcfc90a84d759a6998dfe24186edb"} Jul 08 18:29:47.388261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.388223 2580 scope.go:117] "RemoveContainer" containerID="206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1" Jul 08 18:29:47.396969 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.396946 2580 scope.go:117] "RemoveContainer" containerID="206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1" Jul 08 18:29:47.397200 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:29:47.397182 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1\": container with ID starting with 206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1 not found: ID does not exist" containerID="206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1" Jul 08 18:29:47.397247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.397208 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1"} err="failed to get container status \"206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1\": rpc error: code = NotFound desc = could not find container \"206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1\": container with ID starting with 206ec9c3a6519ee00bf680ac79743c0a506eb3fefc34511355715ab48645b4f1 not found: ID does not exist" Jul 08 18:29:47.434989 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.434968 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-credential-socket\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435004 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-socket\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435023 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-podinfo\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435054 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-certs\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435088 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8djg\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-kube-api-access-q8djg\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435320 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435131 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istiod-ca-cert\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435320 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435238 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:29:47.435320 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435284 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:29:47.435476 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435403 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:29:47.435561 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435172 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-token\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435617 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435597 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:29:47.435660 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435618 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-data\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435660 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435644 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-envoy\") pod \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\" (UID: \"f9e16909-9f6f-4bfa-802a-6ac7657be3ab\") " Jul 08 18:29:47.435902 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435882 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:29:47.436012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.435991 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.436071 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.436009 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.436071 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.436026 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.436071 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.436040 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.436071 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.436053 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.437332 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.437301 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:29:47.437332 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.437321 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-kube-api-access-q8djg" (OuterVolumeSpecName: "kube-api-access-q8djg") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "kube-api-access-q8djg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:29:47.437499 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.437405 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:29:47.437709 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.437694 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9e16909-9f6f-4bfa-802a-6ac7657be3ab" (UID: "f9e16909-9f6f-4bfa-802a-6ac7657be3ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:29:47.536468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.536439 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.536468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.536463 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.536468 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.536474 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.536775 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.536483 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8djg\" (UniqueName: \"kubernetes.io/projected/f9e16909-9f6f-4bfa-802a-6ac7657be3ab-kube-api-access-q8djg\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:29:47.710659 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.710631 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj"] Jul 08 18:29:47.715403 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:47.715381 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ef85937c-openshift-default-866d5c8cd7-wrsdj"] Jul 08 18:29:48.249211 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.249174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:48.691814 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.691783 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:48.714043 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.714014 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx"] Jul 08 18:29:48.715054 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.715028 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e16909-9f6f-4bfa-802a-6ac7657be3ab" containerName="istio-proxy" Jul 08 18:29:48.715276 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.715228 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e16909-9f6f-4bfa-802a-6ac7657be3ab" containerName="istio-proxy" Jul 08 18:29:48.715707 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.715692 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e16909-9f6f-4bfa-802a-6ac7657be3ab" containerName="istio-proxy" Jul 08 18:29:48.723946 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.723685 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.726712 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.726688 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-07d6fd50-gw-openshift-default-dockercfg-srhnm\"" Jul 08 18:29:48.733372 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.733343 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx"] Jul 08 18:29:48.765416 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.765390 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:48.830507 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.830480 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:48.848593 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-data\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.848593 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istiod-ca-cert\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.848873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-credential-socket\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.848873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b8lf\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-kube-api-access-7b8lf\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.848873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-certs\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.848873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-token\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.849067 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-socket\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.849067 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-envoy\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.849067 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.848936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-podinfo\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.870455 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.870427 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e16909-9f6f-4bfa-802a-6ac7657be3ab" path="/var/lib/kubelet/pods/f9e16909-9f6f-4bfa-802a-6ac7657be3ab/volumes" Jul 08 18:29:48.950143 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-socket\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950143 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-envoy\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950143 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-podinfo\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-data\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istiod-ca-cert\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-credential-socket\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b8lf\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-kube-api-access-7b8lf\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950301 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-certs\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-token\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950554 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-socket\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950812 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950784 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-data\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950901 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950857 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-certs\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950957 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950939 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istiod-ca-cert\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.950957 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.950946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-credential-socket\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.952416 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.952396 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-envoy\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.952616 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.952600 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-podinfo\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.958604 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.958585 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-token\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:48.959542 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:48.959522 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b8lf\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-kube-api-access-7b8lf\") pod \"e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:49.036342 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.036316 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:49.174521 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.174499 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx"] Jul 08 18:29:49.176500 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:29:49.176475 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbff22bb_6271_4a1c_a8a1_e3e7acf44f9b.slice/crio-dda198ae482f49c2620b6a4535f2e922d42540db509e4f1141ded18be674fe94 WatchSource:0}: Error finding container dda198ae482f49c2620b6a4535f2e922d42540db509e4f1141ded18be674fe94: Status 404 returned error can't find the container with id dda198ae482f49c2620b6a4535f2e922d42540db509e4f1141ded18be674fe94 Jul 08 18:29:49.178606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.178572 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:29:49.178708 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.178636 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:29:49.178708 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.178667 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:29:49.401882 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.401826 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" event={"ID":"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b","Type":"ContainerStarted","Data":"1d93d82dcca84728e186968070c959e9d840fb8fb9aa483efe527c2a30b3ff4f"} Jul 08 18:29:49.402117 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.401912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" event={"ID":"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b","Type":"ContainerStarted","Data":"dda198ae482f49c2620b6a4535f2e922d42540db509e4f1141ded18be674fe94"} Jul 08 18:29:49.422282 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.422230 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" podStartSLOduration=1.422211615 podStartE2EDuration="1.422211615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:29:49.421981156 +0000 UTC m=+1545.107187926" watchObservedRunningTime="2026-07-08 18:29:49.422211615 +0000 UTC m=+1545.107418384" Jul 08 18:29:49.973811 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:49.973764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:50.037353 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.037307 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:50.043208 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.043185 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:50.079187 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.079155 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:50.407661 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.407629 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:50.408669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.408649 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:29:50.561431 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.561399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:50.660961 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.660872 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:50.762463 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:50.762430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:52.494418 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:52.494335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:54.557502 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:54.557457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:54.717337 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:54.717301 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx"] Jul 08 18:29:54.717573 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:54.717537 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" podUID="dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" containerName="istio-proxy" containerID="cri-o://1d93d82dcca84728e186968070c959e9d840fb8fb9aa483efe527c2a30b3ff4f" gracePeriod=30 Jul 08 18:29:54.745532 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:54.745503 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:54.838805 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:54.838715 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:54.963759 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:54.963709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:29:56.545857 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:29:56.530910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:00.464801 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.464759 2580 generic.go:356] "Generic (PLEG): container finished" podID="dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" containerID="1d93d82dcca84728e186968070c959e9d840fb8fb9aa483efe527c2a30b3ff4f" exitCode=0 Jul 08 18:30:00.465127 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.464827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" event={"ID":"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b","Type":"ContainerDied","Data":"1d93d82dcca84728e186968070c959e9d840fb8fb9aa483efe527c2a30b3ff4f"} Jul 08 18:30:00.465127 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.464923 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" event={"ID":"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b","Type":"ContainerDied","Data":"dda198ae482f49c2620b6a4535f2e922d42540db509e4f1141ded18be674fe94"} Jul 08 18:30:00.465127 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.464943 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dda198ae482f49c2620b6a4535f2e922d42540db509e4f1141ded18be674fe94" Jul 08 18:30:00.480918 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.480895 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:30:00.555817 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.555736 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-certs\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.555817 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.555772 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-envoy\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.555817 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.555801 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-podinfo\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.556080 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.555882 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istiod-ca-cert\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.556080 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.555902 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-socket\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.556080 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.555986 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b8lf\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-kube-api-access-7b8lf\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.556080 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556003 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-credential-socket\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.556080 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556025 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-data\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.556080 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556060 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-token\") pod \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\" (UID: \"dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b\") " Jul 08 18:30:00.556387 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556263 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:00.556448 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556393 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.556448 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556399 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:30:00.556660 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556614 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:00.556660 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.556650 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:00.557385 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.557360 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:00.558152 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.558130 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:00.558305 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.558279 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:30:00.558542 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.558523 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:30:00.558600 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.558549 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-kube-api-access-7b8lf" (OuterVolumeSpecName: "kube-api-access-7b8lf") pod "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" (UID: "dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b"). InnerVolumeSpecName "kube-api-access-7b8lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:30:00.657318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657283 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b8lf\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-kube-api-access-7b8lf\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.657318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657313 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.657318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657325 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.657540 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657333 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.657540 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657341 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.657540 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657348 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.657540 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657356 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:00.657540 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:00.657363 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:01.213414 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.213381 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:01.345043 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.345004 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:01.359495 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.359468 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc"] Jul 08 18:30:01.360022 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.359999 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" containerName="istio-proxy" Jul 08 18:30:01.360022 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.360021 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" containerName="istio-proxy" Jul 08 18:30:01.360218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.360139 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" containerName="istio-proxy" Jul 08 18:30:01.363528 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.363508 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.366048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.366027 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5c5fee5f-openshift-default-dockercfg-rbzxd\"" Jul 08 18:30:01.375628 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.375603 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc"] Jul 08 18:30:01.433110 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.433080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-certs\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-podinfo\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-token\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-envoy\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-data\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464728 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-socket\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istiod-ca-cert\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464863 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-credential-socket\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.464890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.464893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbgxs\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-kube-api-access-mbgxs\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.470602 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.470574 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx" Jul 08 18:30:01.489483 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.489447 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx"] Jul 08 18:30:01.495186 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.495163 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-07d6fd50-gw-openshift-default-6f5c57b89f-q4bjx"] Jul 08 18:30:01.565504 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-socket\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istiod-ca-cert\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-credential-socket\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbgxs\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-kube-api-access-mbgxs\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565824 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-certs\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565824 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-podinfo\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565824 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-token\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565824 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-envoy\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.565824 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-data\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.566173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565922 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-socket\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.566173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.565993 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-credential-socket\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.566173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.566161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-data\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.566328 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.566271 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-certs\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.566328 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.566297 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istiod-ca-cert\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.568232 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.568209 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-envoy\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.568331 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.568275 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-podinfo\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.578276 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.578257 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbgxs\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-kube-api-access-mbgxs\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.578388 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.578372 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-token\") pod \"e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.675560 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.675530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:01.825614 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.825592 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc"] Jul 08 18:30:01.827667 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:30:01.827641 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8968e78f_9d16_4d2f_89f4_b13ac2be87fc.slice/crio-22c3689f2daddac39ab839519ea398260211abd8a77bf935e6d74c1e1d227131 WatchSource:0}: Error finding container 22c3689f2daddac39ab839519ea398260211abd8a77bf935e6d74c1e1d227131: Status 404 returned error can't find the container with id 22c3689f2daddac39ab839519ea398260211abd8a77bf935e6d74c1e1d227131 Jul 08 18:30:01.829730 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.829700 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:30:01.829791 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.829763 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:30:01.829828 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:01.829789 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:30:02.478038 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:02.478001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" event={"ID":"8968e78f-9d16-4d2f-89f4-b13ac2be87fc","Type":"ContainerStarted","Data":"8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31"} Jul 08 18:30:02.478447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:02.478064 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" event={"ID":"8968e78f-9d16-4d2f-89f4-b13ac2be87fc","Type":"ContainerStarted","Data":"22c3689f2daddac39ab839519ea398260211abd8a77bf935e6d74c1e1d227131"} Jul 08 18:30:02.499542 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:02.499501 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" podStartSLOduration=1.499486217 podStartE2EDuration="1.499486217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:30:02.497286658 +0000 UTC m=+1558.182493428" watchObservedRunningTime="2026-07-08 18:30:02.499486217 +0000 UTC m=+1558.184693058" Jul 08 18:30:02.581934 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:02.581764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:02.675815 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:02.675782 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:02.681356 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:02.681324 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:02.870205 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:02.870127 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b" path="/var/lib/kubelet/pods/dbff22bb-6271-4a1c-a8a1-e3e7acf44f9b/volumes" Jul 08 18:30:03.090436 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:03.090373 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:03.189965 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:03.189932 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:03.439610 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:03.439579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:03.483163 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:03.483072 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:03.484046 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:03.484025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:03.508291 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:03.508264 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:04.946054 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:04.946023 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:07.672822 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:07.672787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:08.227093 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:08.227063 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:13.789497 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:13.789404 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc"] Jul 08 18:30:13.790316 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:13.789672 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" podUID="8968e78f-9d16-4d2f-89f4-b13ac2be87fc" containerName="istio-proxy" containerID="cri-o://8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31" gracePeriod=30 Jul 08 18:30:13.818047 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:13.818015 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:13.924052 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:13.924004 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:13.942426 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:13.942403 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:14.090604 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090580 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-credential-socket\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.090764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090614 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-certs\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.090764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090629 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istiod-ca-cert\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.090764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090657 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-token\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.090764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090698 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-podinfo\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.090764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090730 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-data\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.090764 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090750 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-socket\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.091110 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090787 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbgxs\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-kube-api-access-mbgxs\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.091110 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090820 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-envoy\") pod \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\" (UID: \"8968e78f-9d16-4d2f-89f4-b13ac2be87fc\") " Jul 08 18:30:14.091110 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090828 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:14.091110 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.090929 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:14.091329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.091124 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:30:14.091329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.091215 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:14.091329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.091270 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.091329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.091287 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.091329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.091302 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.091974 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.091944 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:14.093070 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.093045 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:30:14.093187 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.093089 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:14.093187 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.093134 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-kube-api-access-mbgxs" (OuterVolumeSpecName: "kube-api-access-mbgxs") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "kube-api-access-mbgxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:30:14.093269 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.093183 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "8968e78f-9d16-4d2f-89f4-b13ac2be87fc" (UID: "8968e78f-9d16-4d2f-89f4-b13ac2be87fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:30:14.192112 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.192057 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.192112 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.192079 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.192112 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.192087 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.192112 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.192095 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.192112 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.192103 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbgxs\" (UniqueName: \"kubernetes.io/projected/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-kube-api-access-mbgxs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.192112 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.192111 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8968e78f-9d16-4d2f-89f4-b13ac2be87fc-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:14.538665 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.538580 2580 generic.go:356] "Generic (PLEG): container finished" podID="8968e78f-9d16-4d2f-89f4-b13ac2be87fc" containerID="8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31" exitCode=0 Jul 08 18:30:14.538665 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.538628 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" Jul 08 18:30:14.538665 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.538641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" event={"ID":"8968e78f-9d16-4d2f-89f4-b13ac2be87fc","Type":"ContainerDied","Data":"8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31"} Jul 08 18:30:14.538986 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.538681 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc" event={"ID":"8968e78f-9d16-4d2f-89f4-b13ac2be87fc","Type":"ContainerDied","Data":"22c3689f2daddac39ab839519ea398260211abd8a77bf935e6d74c1e1d227131"} Jul 08 18:30:14.538986 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.538701 2580 scope.go:117] "RemoveContainer" containerID="8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31" Jul 08 18:30:14.548382 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.548363 2580 scope.go:117] "RemoveContainer" containerID="8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31" Jul 08 18:30:14.548671 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:30:14.548654 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31\": container with ID starting with 8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31 not found: ID does not exist" containerID="8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31" Jul 08 18:30:14.548748 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.548678 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31"} err="failed to get container status \"8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31\": rpc error: code = NotFound desc = could not find container \"8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31\": container with ID starting with 8075e007ce835404d984fd2c19151ae9c91c2a836f87c54ce51dbb3858c6ca31 not found: ID does not exist" Jul 08 18:30:14.562132 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.562109 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc"] Jul 08 18:30:14.567559 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.567538 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5c5fee5f-openshift-default-59b9574c47-gnbfc"] Jul 08 18:30:14.870438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:14.870361 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8968e78f-9d16-4d2f-89f4-b13ac2be87fc" path="/var/lib/kubelet/pods/8968e78f-9d16-4d2f-89f4-b13ac2be87fc/volumes" Jul 08 18:30:16.484721 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:16.484679 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:20.098607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.098560 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:20.158781 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.158741 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:20.287346 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.287307 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j"] Jul 08 18:30:20.288001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.287979 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8968e78f-9d16-4d2f-89f4-b13ac2be87fc" containerName="istio-proxy" Jul 08 18:30:20.288100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.288003 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8968e78f-9d16-4d2f-89f4-b13ac2be87fc" containerName="istio-proxy" Jul 08 18:30:20.288154 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.288141 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8968e78f-9d16-4d2f-89f4-b13ac2be87fc" containerName="istio-proxy" Jul 08 18:30:20.293083 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.293062 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.295498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.295480 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-feb98e91-openshift-default-dockercfg-wnldt\"" Jul 08 18:30:20.302955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.302936 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j"] Jul 08 18:30:20.315444 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.315416 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:20.338701 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-socket\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338814 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-envoy\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338814 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-data\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338814 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-istio-token\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-credential-socket\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da3e1a43-2d56-426d-8186-fd070369b578-istiod-ca-cert\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-certs\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da3e1a43-2d56-426d-8186-fd070369b578-istio-podinfo\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.338955 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.338947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj458\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-kube-api-access-zj458\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.413426 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.413390 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:20.439923 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.439895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da3e1a43-2d56-426d-8186-fd070369b578-istio-podinfo\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440070 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.439927 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj458\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-kube-api-access-zj458\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440070 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.439993 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-socket\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440070 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-envoy\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440236 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-data\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440236 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-istio-token\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440236 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-credential-socket\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440236 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da3e1a43-2d56-426d-8186-fd070369b578-istiod-ca-cert\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440236 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-certs\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440483 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440448 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-socket\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440546 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440501 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-data\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440633 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440611 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-credential-socket\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440683 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-certs\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.440988 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.440970 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da3e1a43-2d56-426d-8186-fd070369b578-istiod-ca-cert\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.442324 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.442296 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-envoy\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.442419 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.442409 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da3e1a43-2d56-426d-8186-fd070369b578-istio-podinfo\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.448640 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.448615 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-istio-token\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.448843 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.448813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj458\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-kube-api-access-zj458\") pod \"e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.604601 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.604570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:20.746349 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.746327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j"] Jul 08 18:30:20.748472 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:30:20.748448 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3e1a43_2d56_426d_8186_fd070369b578.slice/crio-588c061da527d2a1a23857c34396bc9f1530a19dabe3b1006f9d0e891305a236 WatchSource:0}: Error finding container 588c061da527d2a1a23857c34396bc9f1530a19dabe3b1006f9d0e891305a236: Status 404 returned error can't find the container with id 588c061da527d2a1a23857c34396bc9f1530a19dabe3b1006f9d0e891305a236 Jul 08 18:30:20.750541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.750510 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:30:20.750602 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.750576 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:30:20.750638 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:20.750625 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:30:21.413286 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.413246 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:21.521607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.521569 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:21.579957 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.579930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" event={"ID":"da3e1a43-2d56-426d-8186-fd070369b578","Type":"ContainerStarted","Data":"aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487"} Jul 08 18:30:21.580146 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.579975 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" event={"ID":"da3e1a43-2d56-426d-8186-fd070369b578","Type":"ContainerStarted","Data":"588c061da527d2a1a23857c34396bc9f1530a19dabe3b1006f9d0e891305a236"} Jul 08 18:30:21.604960 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.604931 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:21.612302 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.612268 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:21.612485 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.612437 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:21.636365 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.636311 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" podStartSLOduration=1.63629169 podStartE2EDuration="1.63629169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:30:21.604486477 +0000 UTC m=+1577.289693244" watchObservedRunningTime="2026-07-08 18:30:21.63629169 +0000 UTC m=+1577.321498461" Jul 08 18:30:21.748321 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.748242 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:21.999152 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:21.999046 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:22.099362 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:22.099325 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:22.585567 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:22.585537 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:22.586427 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:22.586412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:23.715214 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:23.715180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:25.908428 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:25.908393 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:26.054080 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:26.054039 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j"] Jul 08 18:30:26.054337 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:26.054304 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" podUID="da3e1a43-2d56-426d-8186-fd070369b578" containerName="istio-proxy" containerID="cri-o://aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487" gracePeriod=30 Jul 08 18:30:26.072847 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:26.072795 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:26.183293 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:26.183197 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:26.377340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:26.377307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:27.463003 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:27.462969 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:32.209538 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.209517 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:32.247669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.247605 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-data\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.247669 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.247641 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-certs\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.247861 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.247695 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-socket\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.247861 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.247711 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj458\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-kube-api-access-zj458\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.247861 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.247732 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-istio-token\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.247861 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.247752 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-credential-socket\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.247861 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.247804 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da3e1a43-2d56-426d-8186-fd070369b578-istiod-ca-cert\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.248193 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.248159 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:32.248310 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.248287 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-envoy\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.248378 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.248341 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da3e1a43-2d56-426d-8186-fd070369b578-istio-podinfo\") pod \"da3e1a43-2d56-426d-8186-fd070369b578\" (UID: \"da3e1a43-2d56-426d-8186-fd070369b578\") " Jul 08 18:30:32.248819 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.248562 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:32.248948 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.248853 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-data" (OuterVolumeSpecName: "istio-data") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:32.249209 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.249156 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da3e1a43-2d56-426d-8186-fd070369b578-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:30:32.256681 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.249463 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:32.256681 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.249750 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.256681 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.249777 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.256681 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.249793 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.256681 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.249807 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.256681 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.249849 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da3e1a43-2d56-426d-8186-fd070369b578-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.256681 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.251117 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da3e1a43-2d56-426d-8186-fd070369b578-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:30:32.259566 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.258641 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-kube-api-access-zj458" (OuterVolumeSpecName: "kube-api-access-zj458") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "kube-api-access-zj458". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:30:32.260073 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.260048 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:30:32.260205 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.260071 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-istio-token" (OuterVolumeSpecName: "istio-token") pod "da3e1a43-2d56-426d-8186-fd070369b578" (UID: "da3e1a43-2d56-426d-8186-fd070369b578"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:30:32.351003 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.350972 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj458\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-kube-api-access-zj458\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.351003 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.350999 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da3e1a43-2d56-426d-8186-fd070369b578-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.351003 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.351008 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da3e1a43-2d56-426d-8186-fd070369b578-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.351270 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.351017 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da3e1a43-2d56-426d-8186-fd070369b578-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:30:32.410987 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.410955 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:32.472172 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.472133 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:32.639299 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.639265 2580 generic.go:356] "Generic (PLEG): container finished" podID="da3e1a43-2d56-426d-8186-fd070369b578" containerID="aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487" exitCode=0 Jul 08 18:30:32.639454 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.639330 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" Jul 08 18:30:32.639454 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.639332 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" event={"ID":"da3e1a43-2d56-426d-8186-fd070369b578","Type":"ContainerDied","Data":"aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487"} Jul 08 18:30:32.639454 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.639406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j" event={"ID":"da3e1a43-2d56-426d-8186-fd070369b578","Type":"ContainerDied","Data":"588c061da527d2a1a23857c34396bc9f1530a19dabe3b1006f9d0e891305a236"} Jul 08 18:30:32.639454 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.639429 2580 scope.go:117] "RemoveContainer" containerID="aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487" Jul 08 18:30:32.649089 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.649071 2580 scope.go:117] "RemoveContainer" containerID="aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487" Jul 08 18:30:32.649339 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:30:32.649320 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487\": container with ID starting with aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487 not found: ID does not exist" containerID="aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487" Jul 08 18:30:32.649393 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.649348 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487"} err="failed to get container status \"aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487\": rpc error: code = NotFound desc = could not find container \"aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487\": container with ID starting with aad2919c2f5d041902ebedde40c5d34c7b4eccbc1191459247e2ad8e90376487 not found: ID does not exist" Jul 08 18:30:32.661607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.661583 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j"] Jul 08 18:30:32.665508 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.665486 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-feb98e91-openshift-default-984fc4fb7-9kp9j"] Jul 08 18:30:32.870658 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.870620 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3e1a43-2d56-426d-8186-fd070369b578" path="/var/lib/kubelet/pods/da3e1a43-2d56-426d-8186-fd070369b578/volumes" Jul 08 18:30:32.872175 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.872128 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:32.999469 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:32.999376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:33.098522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:33.098483 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:34.489780 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:34.489745 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:40.752486 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:40.752443 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:40.877578 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:40.877536 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:40.975538 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:40.975506 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:41.076673 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:41.076554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:30:41.517808 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:30:41.517776 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:07.554129 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:07.554094 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:13.590876 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:13.590812 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:13.691869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:13.691814 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:13.792160 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:13.792120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:29.615523 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:29.615479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:35.776363 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:35.776324 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:35.848574 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:35.848513 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:35.936026 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:35.935973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:46.514662 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:46.514617 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:47.320550 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:47.320509 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:53.397255 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:53.397200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:53.456086 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:53.456051 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:53.522374 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:53.522340 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:53.625788 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:53.625754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:53.686672 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:53.686595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:54.448030 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:54.447997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:54.553371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:54.553338 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:54.648437 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:54.648395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:31:56.230819 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:31:56.230786 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:02.820877 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:02.820823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:02.915951 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:02.915919 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:03.197407 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:03.197366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:09.226269 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.226233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:09.403148 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.403111 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:09.503345 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.503256 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:09.798063 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.797973 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm"] Jul 08 18:32:09.798457 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.798439 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da3e1a43-2d56-426d-8186-fd070369b578" containerName="istio-proxy" Jul 08 18:32:09.798551 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.798460 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3e1a43-2d56-426d-8186-fd070369b578" containerName="istio-proxy" Jul 08 18:32:09.798611 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.798599 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="da3e1a43-2d56-426d-8186-fd070369b578" containerName="istio-proxy" Jul 08 18:32:09.801724 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.801699 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.804340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.804316 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cd670168-openshift-default-dockercfg-ng8qh\"" Jul 08 18:32:09.811298 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.811278 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm"] Jul 08 18:32:09.830622 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.830591 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:09.930481 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e70ea6d-7f5c-46c2-be05-a6982085264b-istiod-ca-cert\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930481 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-kube-api-access-qnhcv\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930693 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-envoy\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930693 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-credential-socket\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930693 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-socket\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930794 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-certs\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930794 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-token\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930794 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-data\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:09.930794 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:09.930783 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-podinfo\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032321 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-envoy\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032491 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-credential-socket\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032491 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-socket\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032491 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-certs\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032491 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-token\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032711 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-data\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032711 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032517 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-podinfo\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032711 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e70ea6d-7f5c-46c2-be05-a6982085264b-istiod-ca-cert\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032711 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-kube-api-access-qnhcv\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032947 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032770 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-credential-socket\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.032947 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.032884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-socket\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.033295 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.033271 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-certs\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.033295 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.033287 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-data\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.033428 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.033389 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e70ea6d-7f5c-46c2-be05-a6982085264b-istiod-ca-cert\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.034658 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.034635 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-envoy\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.035027 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.035008 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-podinfo\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.040614 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.040587 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-kube-api-access-qnhcv\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.041130 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.041107 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-token\") pod \"e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.113587 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.113503 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:10.254146 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.254122 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm"] Jul 08 18:32:10.256505 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:32:10.256475 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e70ea6d_7f5c_46c2_be05_a6982085264b.slice/crio-e997bc6946e243fd7132a7f1a121acb997a83386da2bdc2ac4d7aa035d89b3ee WatchSource:0}: Error finding container e997bc6946e243fd7132a7f1a121acb997a83386da2bdc2ac4d7aa035d89b3ee: Status 404 returned error can't find the container with id e997bc6946e243fd7132a7f1a121acb997a83386da2bdc2ac4d7aa035d89b3ee Jul 08 18:32:10.258718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.258684 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:10.258818 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.258761 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:10.258818 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.258806 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:10.753541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:10.753504 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:11.134432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:11.134347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" event={"ID":"6e70ea6d-7f5c-46c2-be05-a6982085264b","Type":"ContainerStarted","Data":"031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff"} Jul 08 18:32:11.134432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:11.134407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" event={"ID":"6e70ea6d-7f5c-46c2-be05-a6982085264b","Type":"ContainerStarted","Data":"e997bc6946e243fd7132a7f1a121acb997a83386da2bdc2ac4d7aa035d89b3ee"} Jul 08 18:32:11.154197 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:11.154142 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" podStartSLOduration=2.154119819 podStartE2EDuration="2.154119819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:32:11.153174968 +0000 UTC m=+1686.838381737" watchObservedRunningTime="2026-07-08 18:32:11.154119819 +0000 UTC m=+1686.839326588" Jul 08 18:32:12.113789 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:12.113754 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:12.118533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:12.118511 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:12.140562 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:12.140535 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:12.141533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:12.141510 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:13.134759 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:13.134719 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:13.234041 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:13.233996 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:13.339608 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:13.339576 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:13.819935 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:13.819888 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:13.920170 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:13.920132 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:17.646667 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:17.646634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:18.323247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:18.323207 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:18.417463 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:18.417427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:18.754490 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:18.754423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:25.047098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:25.047061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:25.101939 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:25.101899 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:25.458771 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:25.458745 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:25.809087 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:25.808986 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:25.911963 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:25.911929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:26.214568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:26.214537 2580 generic.go:356] "Generic (PLEG): container finished" podID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerID="031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff" exitCode=0 Jul 08 18:32:26.214987 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:26.214611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" event={"ID":"6e70ea6d-7f5c-46c2-be05-a6982085264b","Type":"ContainerDied","Data":"031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff"} Jul 08 18:32:26.215044 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:26.215033 2580 scope.go:117] "RemoveContainer" containerID="031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff" Jul 08 18:32:26.216075 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:26.216046 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:26.216151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:26.216134 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:26.216190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:26.216179 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:27.141469 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:27.141396 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:27.221849 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:27.221807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" event={"ID":"6e70ea6d-7f5c-46c2-be05-a6982085264b","Type":"ContainerStarted","Data":"eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659"} Jul 08 18:32:28.114065 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:28.114033 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:28.115315 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:28.115293 2580 patch_prober.go:28] interesting pod/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 08 18:32:28.115386 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:28.115342 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 08 18:32:29.114577 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:29.114541 2580 patch_prober.go:28] interesting pod/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 08 18:32:29.114994 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:29.114613 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 08 18:32:30.114477 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:30.114443 2580 patch_prober.go:28] interesting pod/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 08 18:32:30.114662 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:30.114497 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 08 18:32:31.114088 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:31.114048 2580 patch_prober.go:28] interesting pod/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 08 18:32:31.114291 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:31.114107 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 08 18:32:31.980328 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:31.980293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:32.077604 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.077567 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:32.119494 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.119469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:32.119750 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.119730 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:32.120497 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.120481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:32.239651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.239579 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm"] Jul 08 18:32:32.251107 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.251078 2580 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" secret="" err="secret \"e2e-mt-cd670168-openshift-default-dockercfg-ng8qh\" not found" Jul 08 18:32:32.295511 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.295475 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:32.394656 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:32.394625 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:33.255983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:33.255949 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" containerID="cri-o://eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659" gracePeriod=30 Jul 08 18:32:33.938259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:33.938230 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:38.517521 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.517464 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:38.606538 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606510 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-token\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.606713 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606561 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-data\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.606713 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606585 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e70ea6d-7f5c-46c2-be05-a6982085264b-istiod-ca-cert\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.606713 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606607 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-certs\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.606713 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606653 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-socket\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.606713 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606703 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-kube-api-access-qnhcv\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.607018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606736 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-podinfo\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.607018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606773 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-envoy\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.607018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.606798 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-credential-socket\") pod \"6e70ea6d-7f5c-46c2-be05-a6982085264b\" (UID: \"6e70ea6d-7f5c-46c2-be05-a6982085264b\") " Jul 08 18:32:38.607188 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.607061 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:32:38.607248 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.607209 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.607307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.607065 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e70ea6d-7f5c-46c2-be05-a6982085264b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:32:38.607307 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.607283 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:32:38.607399 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.607302 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:32:38.607626 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.607601 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:32:38.608953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.608926 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:32:38.609158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.609138 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:32:38.609293 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.609277 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-kube-api-access-qnhcv" (OuterVolumeSpecName: "kube-api-access-qnhcv") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "kube-api-access-qnhcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:32:38.609357 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.609328 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e70ea6d-7f5c-46c2-be05-a6982085264b" (UID: "6e70ea6d-7f5c-46c2-be05-a6982085264b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:32:38.708233 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708202 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.708233 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708233 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnhcv\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-kube-api-access-qnhcv\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.708438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708247 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.708438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708258 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.708438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708269 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.708438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708281 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.708438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708294 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e70ea6d-7f5c-46c2-be05-a6982085264b-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:38.708438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:38.708305 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e70ea6d-7f5c-46c2-be05-a6982085264b-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:32:39.118887 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.118830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:39.178592 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.178560 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:39.253604 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.253570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:39.286898 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.286866 2580 generic.go:356] "Generic (PLEG): container finished" podID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerID="eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659" exitCode=0 Jul 08 18:32:39.287045 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.286913 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" Jul 08 18:32:39.287045 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.286935 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" event={"ID":"6e70ea6d-7f5c-46c2-be05-a6982085264b","Type":"ContainerDied","Data":"eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659"} Jul 08 18:32:39.287045 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.286983 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm" event={"ID":"6e70ea6d-7f5c-46c2-be05-a6982085264b","Type":"ContainerDied","Data":"e997bc6946e243fd7132a7f1a121acb997a83386da2bdc2ac4d7aa035d89b3ee"} Jul 08 18:32:39.287045 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.287005 2580 scope.go:117] "RemoveContainer" containerID="eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659" Jul 08 18:32:39.296141 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.296121 2580 scope.go:117] "RemoveContainer" containerID="031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff" Jul 08 18:32:39.304184 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.304170 2580 scope.go:117] "RemoveContainer" containerID="eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659" Jul 08 18:32:39.304449 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:32:39.304426 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659\": container with ID starting with eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659 not found: ID does not exist" containerID="eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659" Jul 08 18:32:39.304506 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.304461 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659"} err="failed to get container status \"eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659\": rpc error: code = NotFound desc = could not find container \"eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659\": container with ID starting with eed6f0089fe6f18f30590ab2ffb97a27b7917eff9400a2cf39deb8fb6f763659 not found: ID does not exist" Jul 08 18:32:39.304506 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.304486 2580 scope.go:117] "RemoveContainer" containerID="031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff" Jul 08 18:32:39.304729 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:32:39.304705 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff\": container with ID starting with 031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff not found: ID does not exist" containerID="031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff" Jul 08 18:32:39.304872 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.304731 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff"} err="failed to get container status \"031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff\": rpc error: code = NotFound desc = could not find container \"031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff\": container with ID starting with 031ae2761fe55bea90e0c542b63a119ed466653d03494affdea8720786454fff not found: ID does not exist" Jul 08 18:32:39.305115 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.305100 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm"] Jul 08 18:32:39.308578 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.308551 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:39.309452 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.309432 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cd670168-openshift-default-c6bd76fc5-26zhm"] Jul 08 18:32:39.494854 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:39.494755 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:40.869318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:40.869286 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" path="/var/lib/kubelet/pods/6e70ea6d-7f5c-46c2-be05-a6982085264b/volumes" Jul 08 18:32:45.613532 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:45.613490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:45.676412 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:45.676364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:52.510691 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.510662 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:52.814428 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.814347 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc"] Jul 08 18:32:52.814980 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.814960 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" Jul 08 18:32:52.814980 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.814979 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" Jul 08 18:32:52.815170 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.815119 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" Jul 08 18:32:52.819565 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.819543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.822163 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.822145 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3ef261af-openshift-default-dockercfg-nlksq\"" Jul 08 18:32:52.830470 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.830447 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc"] Jul 08 18:32:52.836260 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.836235 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:52.937000 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.936950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:52.943002 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.942965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-certs\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd77r\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-kube-api-access-gd77r\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-credential-socket\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943214 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-data\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943214 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943164 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istiod-ca-cert\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943214 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943205 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-envoy\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943311 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-podinfo\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943311 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-token\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:52.943377 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:52.943325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-socket\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd77r\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-kube-api-access-gd77r\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-credential-socket\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-data\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istiod-ca-cert\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-envoy\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-podinfo\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-token\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-socket\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.044698 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-certs\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.045119 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044879 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-credential-socket\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.045119 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.044901 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-data\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.045119 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.045066 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-certs\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.045119 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.045108 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-socket\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.045380 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.045359 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istiod-ca-cert\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.046892 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.046868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-envoy\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.047093 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.047073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-podinfo\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.053662 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.053639 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-token\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.053749 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.053669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd77r\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-kube-api-access-gd77r\") pod \"e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.130567 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.130510 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:53.279579 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.279552 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc"] Jul 08 18:32:53.281819 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:32:53.281793 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6958c3b_49ca_4a0b_88d9_bedb6d1b9824.slice/crio-9fe3f46d6f03ec746773eee57ccb05c2a82e6a03e5e26df97950055e2268d32c WatchSource:0}: Error finding container 9fe3f46d6f03ec746773eee57ccb05c2a82e6a03e5e26df97950055e2268d32c: Status 404 returned error can't find the container with id 9fe3f46d6f03ec746773eee57ccb05c2a82e6a03e5e26df97950055e2268d32c Jul 08 18:32:53.284004 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.283973 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:53.284082 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.284042 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:53.284082 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.284068 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:32:53.360566 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.360536 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" event={"ID":"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824","Type":"ContainerStarted","Data":"942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8"} Jul 08 18:32:53.360661 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.360593 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" event={"ID":"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824","Type":"ContainerStarted","Data":"9fe3f46d6f03ec746773eee57ccb05c2a82e6a03e5e26df97950055e2268d32c"} Jul 08 18:32:53.384637 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:53.384540 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" podStartSLOduration=1.384524639 podStartE2EDuration="1.384524639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:32:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:32:53.383261737 +0000 UTC m=+1729.068468505" watchObservedRunningTime="2026-07-08 18:32:53.384524639 +0000 UTC m=+1729.069731449" Jul 08 18:32:54.131645 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:54.131609 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:54.136536 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:54.136513 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:54.369145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:54.369116 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:54.370019 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:54.370002 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:32:55.298029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:55.297989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:55.374360 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:55.374324 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:55.436819 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:55.436780 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:55.537217 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:55.537171 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:55.933018 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:55.932985 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:56.032760 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:56.032724 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:57.485647 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:57.485604 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:57.588179 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:57.588141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:32:59.806823 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:32:59.806791 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:09.493374 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:09.491996 2580 patch_prober.go:28] interesting pod/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:34148->10.132.0.58:15021: read: connection reset by peer" start-of-body= Jul 08 18:33:09.493374 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:09.492071 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:34148->10.132.0.58:15021: read: connection reset by peer" Jul 08 18:33:10.460610 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:10.460579 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerID="942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8" exitCode=0 Jul 08 18:33:10.460757 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:10.460652 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" event={"ID":"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824","Type":"ContainerDied","Data":"942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8"} Jul 08 18:33:10.461058 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:10.461045 2580 scope.go:117] "RemoveContainer" containerID="942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8" Jul 08 18:33:10.462149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:10.462119 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:33:10.462225 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:10.462206 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:33:10.462262 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:10.462247 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:33:11.467802 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:11.467765 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" event={"ID":"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824","Type":"ContainerStarted","Data":"a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0"} Jul 08 18:33:12.130707 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:12.130671 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:33:12.130953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:12.130924 2580 patch_prober.go:28] interesting pod/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 08 18:33:12.131198 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:12.131172 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 08 18:33:13.131953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:13.131911 2580 patch_prober.go:28] interesting pod/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 08 18:33:13.132484 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:13.131980 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 08 18:33:13.988773 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:13.988737 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:14.131431 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:14.131399 2580 patch_prober.go:28] interesting pod/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 08 18:33:14.131718 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:14.131689 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 08 18:33:15.131478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:15.131438 2580 patch_prober.go:28] interesting pod/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 08 18:33:15.132016 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:15.131509 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 08 18:33:16.135524 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:16.135492 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:33:16.135951 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:16.135869 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:33:16.137168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:16.137150 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:33:16.722622 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:16.722566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:17.034403 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:17.034313 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:32.226403 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:32.226360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:32.628256 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:32.628172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:38.310597 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:38.310556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:33:43.389122 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:33:43.389060 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:04.878798 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:04.878762 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:34:04.882723 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:04.882701 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:34:19.910730 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:19.910693 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:20.824752 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:20.824710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:26.607394 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:26.607350 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:32.595220 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:32.595176 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:32.653646 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:32.653611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:32.853254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:32.853169 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc"] Jul 08 18:34:32.853455 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:32.853428 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" containerID="cri-o://a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0" gracePeriod=30 Jul 08 18:34:32.877739 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:32.877707 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:34.151763 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:34.151731 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:34.252471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:34.252441 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:34.352719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:34.352687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:35.098325 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:35.098296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:35.197697 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:35.197663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:35.297890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:35.297859 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:36.676012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:36.675973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:38.121178 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.121120 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:34:38.219031 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.218998 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-envoy\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219040 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-podinfo\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219108 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istiod-ca-cert\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219138 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-certs\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219174 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-credential-socket\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219440 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219226 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd77r\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-kube-api-access-gd77r\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219440 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219249 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-socket\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219440 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219315 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-data\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219440 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219342 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-token\") pod \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\" (UID: \"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824\") " Jul 08 18:34:38.219649 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219446 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:34:38.219649 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219493 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:34:38.219649 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219626 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:34:38.219815 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219699 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.219815 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219779 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.219937 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.219875 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:34:38.220366 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.220344 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:34:38.221316 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.221295 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:34:38.222023 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.221994 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:34:38.222023 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.222010 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:34:38.222155 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.222026 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-kube-api-access-gd77r" (OuterVolumeSpecName: "kube-api-access-gd77r") pod "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" (UID: "c6958c3b-49ca-4a0b-88d9-bedb6d1b9824"). InnerVolumeSpecName "kube-api-access-gd77r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:34:38.320786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.320748 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.320786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.320774 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.320786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.320782 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.320786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.320790 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.320786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.320799 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.321216 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.320806 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd77r\" (UniqueName: \"kubernetes.io/projected/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-kube-api-access-gd77r\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.321216 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.320817 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:34:38.929374 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.929341 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerID="a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0" exitCode=0 Jul 08 18:34:38.929606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.929390 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" Jul 08 18:34:38.929606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.929432 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" event={"ID":"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824","Type":"ContainerDied","Data":"a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0"} Jul 08 18:34:38.929606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.929490 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc" event={"ID":"c6958c3b-49ca-4a0b-88d9-bedb6d1b9824","Type":"ContainerDied","Data":"9fe3f46d6f03ec746773eee57ccb05c2a82e6a03e5e26df97950055e2268d32c"} Jul 08 18:34:38.929606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.929514 2580 scope.go:117] "RemoveContainer" containerID="a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0" Jul 08 18:34:38.946481 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.946459 2580 scope.go:117] "RemoveContainer" containerID="942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8" Jul 08 18:34:38.952869 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.952827 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc"] Jul 08 18:34:38.954763 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.954744 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3ef261af-openshift-default-687d7f657f-n48zc"] Jul 08 18:34:38.957050 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.957037 2580 scope.go:117] "RemoveContainer" containerID="a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0" Jul 08 18:34:38.957326 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:34:38.957305 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0\": container with ID starting with a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0 not found: ID does not exist" containerID="a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0" Jul 08 18:34:38.957411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.957334 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0"} err="failed to get container status \"a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0\": rpc error: code = NotFound desc = could not find container \"a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0\": container with ID starting with a03d064c8c8379a9e8d722e1e3840bc811a90b07e20d069bc41ff99d2ccae2b0 not found: ID does not exist" Jul 08 18:34:38.957411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.957354 2580 scope.go:117] "RemoveContainer" containerID="942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8" Jul 08 18:34:38.957638 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:34:38.957619 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8\": container with ID starting with 942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8 not found: ID does not exist" containerID="942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8" Jul 08 18:34:38.957701 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:38.957641 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8"} err="failed to get container status \"942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8\": rpc error: code = NotFound desc = could not find container \"942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8\": container with ID starting with 942ea5cbe00eb6bda89f6c2965ef636b50880433a86a22c354525e23484b87d8 not found: ID does not exist" Jul 08 18:34:40.869769 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:40.869726 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" path="/var/lib/kubelet/pods/c6958c3b-49ca-4a0b-88d9-bedb6d1b9824/volumes" Jul 08 18:34:42.742073 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:42.742040 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:42.845425 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:42.845388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:43.408769 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:43.408731 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:49.212291 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:49.212252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:49.269352 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:49.269319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:49.337193 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:49.337153 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:49.445072 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:49.445041 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:34:59.976536 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:34:59.976500 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:00.355048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:00.354966 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:21.394753 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:21.394721 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:27.340719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.340631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:27.441368 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.441330 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:27.671158 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.671104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:27.956889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.956801 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc"] Jul 08 18:35:27.957438 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957423 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" Jul 08 18:35:27.957482 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957442 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" Jul 08 18:35:27.957482 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957465 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" Jul 08 18:35:27.957482 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957474 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" Jul 08 18:35:27.957582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957493 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" Jul 08 18:35:27.957582 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957502 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" Jul 08 18:35:27.957648 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957589 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e70ea6d-7f5c-46c2-be05-a6982085264b" containerName="istio-proxy" Jul 08 18:35:27.957648 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.957603 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" Jul 08 18:35:27.961109 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.961088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:27.963564 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.963549 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-94205f-openshift-default-dockercfg-5kknc\"" Jul 08 18:35:27.971498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.971469 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:27.972011 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:27.971973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc"] Jul 08 18:35:28.077176 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-credential-socket\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077348 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpcmv\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-kube-api-access-fpcmv\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077348 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b52970-0c00-467d-afcf-50858d8505db-istiod-ca-cert\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077425 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-socket\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077459 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077429 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b52970-0c00-467d-afcf-50858d8505db-istio-podinfo\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-envoy\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077535 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-istio-token\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-data\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.077605 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.077562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-certs\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178376 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b52970-0c00-467d-afcf-50858d8505db-istiod-ca-cert\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178376 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-socket\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b52970-0c00-467d-afcf-50858d8505db-istio-podinfo\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178443 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-envoy\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-istio-token\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-data\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-certs\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-credential-socket\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.178651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.178603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpcmv\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-kube-api-access-fpcmv\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.179069 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.179047 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-socket\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.179151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.179117 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-credential-socket\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.179151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.179144 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-certs\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.179267 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.179221 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b52970-0c00-467d-afcf-50858d8505db-istiod-ca-cert\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.179373 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.179354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-data\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.180944 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.180926 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-envoy\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.181160 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.181142 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b52970-0c00-467d-afcf-50858d8505db-istio-podinfo\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.186389 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.186368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-istio-token\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.186389 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.186387 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpcmv\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-kube-api-access-fpcmv\") pod \"e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.273710 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.273624 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:28.438089 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.435751 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc"] Jul 08 18:35:28.439889 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.439865 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:28.442413 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:35:28.442391 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b52970_0c00_467d_afcf_50858d8505db.slice/crio-81b08af7afb6804f178227143f4be0e83ef0a0ba2fbb791b8cb15a313c44b777 WatchSource:0}: Error finding container 81b08af7afb6804f178227143f4be0e83ef0a0ba2fbb791b8cb15a313c44b777: Status 404 returned error can't find the container with id 81b08af7afb6804f178227143f4be0e83ef0a0ba2fbb791b8cb15a313c44b777 Jul 08 18:35:28.444439 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.444409 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:28.444517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.444471 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:28.444517 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:28.444498 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:29.181609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:29.181575 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" event={"ID":"88b52970-0c00-467d-afcf-50858d8505db","Type":"ContainerStarted","Data":"8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb"} Jul 08 18:35:29.181816 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:29.181633 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" event={"ID":"88b52970-0c00-467d-afcf-50858d8505db","Type":"ContainerStarted","Data":"81b08af7afb6804f178227143f4be0e83ef0a0ba2fbb791b8cb15a313c44b777"} Jul 08 18:35:29.207797 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:29.207747 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" podStartSLOduration=2.207726741 podStartE2EDuration="2.207726741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:35:29.204084432 +0000 UTC m=+1884.889291202" watchObservedRunningTime="2026-07-08 18:35:29.207726741 +0000 UTC m=+1884.892933512" Jul 08 18:35:29.274303 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:29.274270 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:29.279135 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:29.279101 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:30.186239 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:30.186209 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:30.187230 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:30.187214 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:31.226846 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:31.226808 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:31.316967 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:31.316929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:31.383542 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:31.383505 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:31.821621 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:31.821578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:31.929918 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:31.929874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:32.082401 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:32.082311 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:35.654113 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:35.654077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:35.995147 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:35.995054 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:36.277304 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.277211 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld"] Jul 08 18:35:36.278003 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.277984 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6958c3b-49ca-4a0b-88d9-bedb6d1b9824" containerName="istio-proxy" Jul 08 18:35:36.281810 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.281787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.284301 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.284284 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-44b28a-openshift-default-dockercfg-rqhhx\"" Jul 08 18:35:36.291573 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.291549 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld"] Jul 08 18:35:36.303106 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.303077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:36.454617 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-podinfo\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.454821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghl2q\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-kube-api-access-ghl2q\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.454821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454665 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-envoy\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.454821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-socket\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.454821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454733 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-token\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.454821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-data\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.454821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-certs\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.454821 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-credential-socket\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.455146 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.454939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istiod-ca-cert\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.555903 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.555810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghl2q\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-kube-api-access-ghl2q\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.555903 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.555882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-envoy\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.555908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-socket\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.555932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-token\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.555954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-data\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.555983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-certs\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.556040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-credential-socket\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.556072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istiod-ca-cert\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.556128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-podinfo\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556506 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.556450 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-socket\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556506 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.556480 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-data\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556633 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.556610 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-credential-socket\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.556691 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.556638 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-certs\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.557122 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.557097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istiod-ca-cert\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.558318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.558297 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-envoy\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.558588 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.558572 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-podinfo\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.564029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.564008 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-token\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.564179 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.564160 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghl2q\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-kube-api-access-ghl2q\") pod \"e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.592587 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.592554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:36.594394 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.594370 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:36.742570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.742545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld"] Jul 08 18:35:36.743747 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:35:36.743724 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8231cd0f_551c_4ca9_bc91_40b5ecc3c1b0.slice/crio-d99ab46be28b20c426e2b4883f1d90056d7c47a37a1484b2a43204442a07aa8f WatchSource:0}: Error finding container d99ab46be28b20c426e2b4883f1d90056d7c47a37a1484b2a43204442a07aa8f: Status 404 returned error can't find the container with id d99ab46be28b20c426e2b4883f1d90056d7c47a37a1484b2a43204442a07aa8f Jul 08 18:35:36.745975 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.745948 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:36.746036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.746010 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:36.746073 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:36.746038 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:37.226224 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:37.226188 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" event={"ID":"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0","Type":"ContainerStarted","Data":"9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8"} Jul 08 18:35:37.226432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:37.226270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" event={"ID":"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0","Type":"ContainerStarted","Data":"d99ab46be28b20c426e2b4883f1d90056d7c47a37a1484b2a43204442a07aa8f"} Jul 08 18:35:37.250855 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:37.250780 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" podStartSLOduration=1.250760599 podStartE2EDuration="1.250760599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:35:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:35:37.247719587 +0000 UTC m=+1892.932926357" watchObservedRunningTime="2026-07-08 18:35:37.250760599 +0000 UTC m=+1892.935967370" Jul 08 18:35:37.595512 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:37.595421 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:37.600234 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:37.600208 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:38.231902 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:38.231869 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:38.232777 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:38.232758 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:39.179238 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:39.179202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:39.260151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:39.260117 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:39.317789 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:39.317733 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:39.660853 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:39.660798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:39.971553 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:39.971454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:40.072618 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:40.072577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:43.710245 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:43.710201 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:46.227983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:46.227949 2580 patch_prober.go:28] interesting pod/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:35:46.228340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:46.228002 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:35:47.292149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:47.292064 2580 generic.go:356] "Generic (PLEG): container finished" podID="88b52970-0c00-467d-afcf-50858d8505db" containerID="8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb" exitCode=0 Jul 08 18:35:47.292149 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:47.292138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" event={"ID":"88b52970-0c00-467d-afcf-50858d8505db","Type":"ContainerDied","Data":"8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb"} Jul 08 18:35:47.292590 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:47.292551 2580 scope.go:117] "RemoveContainer" containerID="8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb" Jul 08 18:35:47.293679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:47.293645 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:47.293804 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:47.293735 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:47.293804 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:47.293782 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:48.298892 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:48.298853 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" event={"ID":"88b52970-0c00-467d-afcf-50858d8505db","Type":"ContainerStarted","Data":"b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357"} Jul 08 18:35:49.275499 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:49.275037 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:49.275499 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:49.275178 2580 patch_prober.go:28] interesting pod/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 08 18:35:49.275499 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:49.275223 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 08 18:35:50.274557 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:50.274515 2580 patch_prober.go:28] interesting pod/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 08 18:35:50.275116 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:50.274591 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 08 18:35:51.274727 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:51.274690 2580 patch_prober.go:28] interesting pod/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 08 18:35:51.275145 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:51.274749 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 08 18:35:52.274449 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:52.274404 2580 patch_prober.go:28] interesting pod/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 08 18:35:52.274697 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:52.274472 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 08 18:35:52.840273 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:52.840242 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:52.939541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:52.939495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:53.277645 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:53.277613 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:53.278030 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:53.278006 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:53.278668 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:53.278654 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:35:53.617583 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:53.616240 2580 patch_prober.go:28] interesting pod/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:47040->10.132.0.60:15021: read: connection reset by peer" start-of-body= Jul 08 18:35:53.617583 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:53.616315 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:47040->10.132.0.60:15021: read: connection reset by peer" Jul 08 18:35:54.335004 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:54.334973 2580 generic.go:356] "Generic (PLEG): container finished" podID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerID="9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8" exitCode=0 Jul 08 18:35:54.335495 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:54.335044 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" event={"ID":"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0","Type":"ContainerDied","Data":"9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8"} Jul 08 18:35:54.335564 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:54.335556 2580 scope.go:117] "RemoveContainer" containerID="9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8" Jul 08 18:35:54.336520 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:54.336495 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:54.336575 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:54.336559 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:54.336620 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:54.336592 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:35:55.361144 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:55.361099 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" event={"ID":"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0","Type":"ContainerStarted","Data":"295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610"} Jul 08 18:35:55.594867 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:55.594811 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:55.595091 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:55.595069 2580 patch_prober.go:28] interesting pod/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 18:35:55.595146 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:55.595115 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 18:35:56.595462 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:56.595423 2580 patch_prober.go:28] interesting pod/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 18:35:56.595888 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:56.595483 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 18:35:57.595187 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:57.595153 2580 patch_prober.go:28] interesting pod/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 18:35:57.595370 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:57.595210 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 18:35:57.647534 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:57.647491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:57.746659 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:57.746615 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:58.595702 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:58.595663 2580 patch_prober.go:28] interesting pod/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 18:35:58.596012 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:58.595724 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 18:35:59.478137 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:59.478106 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:35:59.598498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:59.598470 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:59.598737 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:59.598719 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:35:59.599404 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:35:59.599385 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:36:00.800185 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:00.800156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:00.901222 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:00.901190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:05.004043 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:05.004014 2580 scope.go:117] "RemoveContainer" containerID="1d93d82dcca84728e186968070c959e9d840fb8fb9aa483efe527c2a30b3ff4f" Jul 08 18:36:22.858257 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:22.858212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:54.705538 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:54.705499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:54.762964 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:54.762935 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:57.725651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:57.725554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:58.685120 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:58.685080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:58.904781 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:58.904747 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:59.343086 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:59.343056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:59.440658 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:59.440618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:59.543089 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:59.543056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:36:59.648319 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:36:59.648287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:04.423173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:04.423134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:18.031014 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:18.030971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:32.989471 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:32.989437 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:41.354223 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:41.354192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:41.656213 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:41.656182 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc"] Jul 08 18:37:41.656458 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:41.656434 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" containerID="cri-o://b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357" gracePeriod=30 Jul 08 18:37:42.451722 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:42.451691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:42.550503 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:42.550470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:42.651019 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:42.650910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:42.752786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:42.752701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:46.921133 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.921072 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:37:46.939059 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.939030 2580 generic.go:356] "Generic (PLEG): container finished" podID="88b52970-0c00-467d-afcf-50858d8505db" containerID="b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357" exitCode=0 Jul 08 18:37:46.939199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.939079 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" Jul 08 18:37:46.939199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.939093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" event={"ID":"88b52970-0c00-467d-afcf-50858d8505db","Type":"ContainerDied","Data":"b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357"} Jul 08 18:37:46.939199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.939142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc" event={"ID":"88b52970-0c00-467d-afcf-50858d8505db","Type":"ContainerDied","Data":"81b08af7afb6804f178227143f4be0e83ef0a0ba2fbb791b8cb15a313c44b777"} Jul 08 18:37:46.939199 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.939169 2580 scope.go:117] "RemoveContainer" containerID="b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357" Jul 08 18:37:46.951827 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.951805 2580 scope.go:117] "RemoveContainer" containerID="8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb" Jul 08 18:37:46.960371 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.960342 2580 scope.go:117] "RemoveContainer" containerID="b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357" Jul 08 18:37:46.960657 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:37:46.960638 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357\": container with ID starting with b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357 not found: ID does not exist" containerID="b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357" Jul 08 18:37:46.960716 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.960665 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357"} err="failed to get container status \"b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357\": rpc error: code = NotFound desc = could not find container \"b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357\": container with ID starting with b24a8263016cbd3b016943c4883421515021b677624c92c574af240544e74357 not found: ID does not exist" Jul 08 18:37:46.960716 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.960681 2580 scope.go:117] "RemoveContainer" containerID="8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb" Jul 08 18:37:46.960939 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:37:46.960924 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb\": container with ID starting with 8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb not found: ID does not exist" containerID="8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb" Jul 08 18:37:46.960989 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:46.960943 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb"} err="failed to get container status \"8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb\": rpc error: code = NotFound desc = could not find container \"8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb\": container with ID starting with 8d344d9b29fb726e706a06fe817caa92ce4bfa28931adb2a88b71c52adcc4ddb not found: ID does not exist" Jul 08 18:37:47.007104 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007083 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-data\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007249 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007124 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpcmv\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-kube-api-access-fpcmv\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007249 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007147 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b52970-0c00-467d-afcf-50858d8505db-istio-podinfo\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007249 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007198 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-certs\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007249 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007215 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-envoy\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007249 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007247 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b52970-0c00-467d-afcf-50858d8505db-istiod-ca-cert\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007346 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-istio-token\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007383 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-credential-socket\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007433 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-socket\") pod \"88b52970-0c00-467d-afcf-50858d8505db\" (UID: \"88b52970-0c00-467d-afcf-50858d8505db\") " Jul 08 18:37:47.007796 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007732 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:47.007796 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007744 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:47.007943 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007899 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.007943 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007915 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:47.007943 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.007920 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.008104 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.008009 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-data" (OuterVolumeSpecName: "istio-data") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:47.008217 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.008190 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88b52970-0c00-467d-afcf-50858d8505db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:37:47.009510 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.009489 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88b52970-0c00-467d-afcf-50858d8505db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:37:47.009632 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.009607 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-istio-token" (OuterVolumeSpecName: "istio-token") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:37:47.009791 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.009766 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:47.009791 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.009786 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-kube-api-access-fpcmv" (OuterVolumeSpecName: "kube-api-access-fpcmv") pod "88b52970-0c00-467d-afcf-50858d8505db" (UID: "88b52970-0c00-467d-afcf-50858d8505db"). InnerVolumeSpecName "kube-api-access-fpcmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:37:47.108965 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.108935 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.108965 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.108959 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b52970-0c00-467d-afcf-50858d8505db-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.108965 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.108967 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.108965 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.108976 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.109226 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.108984 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b52970-0c00-467d-afcf-50858d8505db-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.109226 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.108993 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpcmv\" (UniqueName: \"kubernetes.io/projected/88b52970-0c00-467d-afcf-50858d8505db-kube-api-access-fpcmv\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.109226 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.109001 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b52970-0c00-467d-afcf-50858d8505db-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:47.262403 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.262373 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc"] Jul 08 18:37:47.267605 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:47.267574 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-94205f-openshift-default-64494b69b5-22gfc"] Jul 08 18:37:48.451457 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:48.451421 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:48.551079 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:48.551048 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:48.703927 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:48.703828 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld"] Jul 08 18:37:48.704128 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:48.704097 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" containerID="cri-o://295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610" gracePeriod=30 Jul 08 18:37:48.719387 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:48.719357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:48.869745 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:48.869714 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88b52970-0c00-467d-afcf-50858d8505db" path="/var/lib/kubelet/pods/88b52970-0c00-467d-afcf-50858d8505db/volumes" Jul 08 18:37:49.323989 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.323953 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:49.612497 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.612424 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br"] Jul 08 18:37:49.613024 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.613006 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" Jul 08 18:37:49.613100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.613027 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" Jul 08 18:37:49.613100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.613045 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" Jul 08 18:37:49.613100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.613052 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" Jul 08 18:37:49.613200 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.613121 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" Jul 08 18:37:49.617591 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.617573 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.619982 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.619963 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-34c6f7-openshift-default-dockercfg-stf84\"" Jul 08 18:37:49.627224 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.627205 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br"] Jul 08 18:37:49.640715 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.640686 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:49.733106 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733076 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608be071-c1c2-4434-bd9e-353dd96db76e-istio-podinfo\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733304 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608be071-c1c2-4434-bd9e-353dd96db76e-istiod-ca-cert\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733304 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733149 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-credential-socket\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733304 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-socket\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733304 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk9v9\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-kube-api-access-bk9v9\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733535 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-certs\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733535 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-istio-token\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733535 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-data\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.733535 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.733500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-envoy\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.834856 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.834796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-istio-token\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.834875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-data\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.834939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-envoy\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.834971 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608be071-c1c2-4434-bd9e-353dd96db76e-istio-podinfo\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835036 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608be071-c1c2-4434-bd9e-353dd96db76e-istiod-ca-cert\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835284 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-credential-socket\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835284 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-socket\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835284 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk9v9\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-kube-api-access-bk9v9\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835284 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835278 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-certs\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835489 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835371 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-data\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835489 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835446 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-credential-socket\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835609 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-socket\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835706 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835679 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-certs\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.835791 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.835738 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608be071-c1c2-4434-bd9e-353dd96db76e-istiod-ca-cert\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.837640 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.837609 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-envoy\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.837888 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.837871 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608be071-c1c2-4434-bd9e-353dd96db76e-istio-podinfo\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.843600 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.843569 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-istio-token\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.843686 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.843660 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk9v9\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-kube-api-access-bk9v9\") pod \"e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:49.929443 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:49.929410 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:50.041591 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.041554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:50.080868 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.080826 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br"] Jul 08 18:37:50.082736 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:37:50.082708 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608be071_c1c2_4434_bd9e_353dd96db76e.slice/crio-9b1d83a0c1c35eb9c07bfb43cf71058cb72c41787b8156692f83ec74c00e9f39 WatchSource:0}: Error finding container 9b1d83a0c1c35eb9c07bfb43cf71058cb72c41787b8156692f83ec74c00e9f39: Status 404 returned error can't find the container with id 9b1d83a0c1c35eb9c07bfb43cf71058cb72c41787b8156692f83ec74c00e9f39 Jul 08 18:37:50.084963 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.084925 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:37:50.085033 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.085005 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:37:50.085085 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.085052 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:37:50.962029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.961998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" event={"ID":"608be071-c1c2-4434-bd9e-353dd96db76e","Type":"ContainerStarted","Data":"feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011"} Jul 08 18:37:50.962442 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.962042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" event={"ID":"608be071-c1c2-4434-bd9e-353dd96db76e","Type":"ContainerStarted","Data":"9b1d83a0c1c35eb9c07bfb43cf71058cb72c41787b8156692f83ec74c00e9f39"} Jul 08 18:37:50.982883 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:50.982820 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" podStartSLOduration=1.9828032580000001 podStartE2EDuration="1.982803258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:37:50.982036564 +0000 UTC m=+2026.667243346" watchObservedRunningTime="2026-07-08 18:37:50.982803258 +0000 UTC m=+2026.668010029" Jul 08 18:37:51.930473 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:51.930441 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:51.935247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:51.935221 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:51.968395 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:51.968375 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:51.969098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:51.969080 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:37:52.897917 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:52.897879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:52.984878 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:52.984844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:53.045422 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.045388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:53.251957 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.251862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:53.442346 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.442311 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:53.543168 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.543080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:53.965049 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.965028 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:37:53.983029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.983005 2580 generic.go:356] "Generic (PLEG): container finished" podID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerID="295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610" exitCode=0 Jul 08 18:37:53.983161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.983051 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" Jul 08 18:37:53.983161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.983059 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" event={"ID":"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0","Type":"ContainerDied","Data":"295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610"} Jul 08 18:37:53.983161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.983112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld" event={"ID":"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0","Type":"ContainerDied","Data":"d99ab46be28b20c426e2b4883f1d90056d7c47a37a1484b2a43204442a07aa8f"} Jul 08 18:37:53.983161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.983145 2580 scope.go:117] "RemoveContainer" containerID="295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610" Jul 08 18:37:53.993866 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:53.993827 2580 scope.go:117] "RemoveContainer" containerID="9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8" Jul 08 18:37:54.002415 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.002401 2580 scope.go:117] "RemoveContainer" containerID="295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610" Jul 08 18:37:54.002734 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:37:54.002706 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610\": container with ID starting with 295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610 not found: ID does not exist" containerID="295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610" Jul 08 18:37:54.002866 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.002741 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610"} err="failed to get container status \"295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610\": rpc error: code = NotFound desc = could not find container \"295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610\": container with ID starting with 295f3424d1ab6612c6edee8622af89ce64ec437b2df993d52922d450fbaf5610 not found: ID does not exist" Jul 08 18:37:54.002866 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.002760 2580 scope.go:117] "RemoveContainer" containerID="9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8" Jul 08 18:37:54.003056 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:37:54.003037 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8\": container with ID starting with 9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8 not found: ID does not exist" containerID="9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8" Jul 08 18:37:54.003112 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.003069 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8"} err="failed to get container status \"9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8\": rpc error: code = NotFound desc = could not find container \"9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8\": container with ID starting with 9d83b392bedcca70d3583fccc5e5169724233282efbbb03c3edff1a84ec66ec8 not found: ID does not exist" Jul 08 18:37:54.076738 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076670 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghl2q\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-kube-api-access-ghl2q\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.076738 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076707 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-envoy\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.076983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076737 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-socket\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.076983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076773 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-certs\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.076983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076803 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-credential-socket\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.076983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076827 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istiod-ca-cert\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.076983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076913 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-podinfo\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.076983 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.076958 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-data\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.077311 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077044 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-token\") pod \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\" (UID: \"8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0\") " Jul 08 18:37:54.077311 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077086 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:54.077311 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077195 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:54.077311 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077199 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:54.077522 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077307 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:37:54.077576 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077555 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.077642 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077576 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.077642 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077592 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.077642 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.077605 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.078040 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.078014 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-data" (OuterVolumeSpecName: "istio-data") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:54.079260 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.079235 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-token" (OuterVolumeSpecName: "istio-token") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:37:54.079411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.079387 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:54.079411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.079401 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-kube-api-access-ghl2q" (OuterVolumeSpecName: "kube-api-access-ghl2q") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "kube-api-access-ghl2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:37:54.079536 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.079438 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" (UID: "8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:37:54.178151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.178120 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.178151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.178149 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.178151 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.178158 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.178432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.178171 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghl2q\" (UniqueName: \"kubernetes.io/projected/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-kube-api-access-ghl2q\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.178432 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.178185 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:37:54.305414 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.305389 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld"] Jul 08 18:37:54.309953 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.309931 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-44b28a-openshift-default-5df59457d5-rk7ld"] Jul 08 18:37:54.871075 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:54.871046 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" path="/var/lib/kubelet/pods/8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0/volumes" Jul 08 18:37:57.408293 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:57.408252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:57.895059 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:57.895021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:58.180699 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.180620 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497"] Jul 08 18:37:58.181295 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.181278 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" Jul 08 18:37:58.181368 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.181297 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" Jul 08 18:37:58.181436 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.181424 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" Jul 08 18:37:58.181492 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.181441 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" Jul 08 18:37:58.181492 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.181454 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="88b52970-0c00-467d-afcf-50858d8505db" containerName="istio-proxy" Jul 08 18:37:58.186290 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.186268 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.191023 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.191002 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-3689d4-openshift-default-dockercfg-qwgpz\"" Jul 08 18:37:58.203705 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.203657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497"] Jul 08 18:37:58.240888 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.240854 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:58.316350 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-socket\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316350 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-certs\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316574 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316411 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-credential-socket\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316574 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsh5w\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-kube-api-access-tsh5w\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316574 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316449 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-token\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316574 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316515 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-podinfo\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316717 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316577 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3786c50-d023-4b42-be71-f42a8f9e3119-istiod-ca-cert\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316717 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-data\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.316717 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.316651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-envoy\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.417684 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.417646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-credential-socket\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.417695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsh5w\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-kube-api-access-tsh5w\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.417723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-token\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.417764 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-podinfo\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.417889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3786c50-d023-4b42-be71-f42a8f9e3119-istiod-ca-cert\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.417921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-data\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.417975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-envoy\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.418103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-credential-socket\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418180 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.418162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-socket\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418623 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.418202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-certs\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418623 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.418380 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-data\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418623 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.418519 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-socket\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418623 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.418587 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-certs\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.418802 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.418716 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3786c50-d023-4b42-be71-f42a8f9e3119-istiod-ca-cert\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.420227 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.420203 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-envoy\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.420624 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.420607 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-podinfo\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.426047 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.426018 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-token\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.426133 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.426095 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsh5w\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-kube-api-access-tsh5w\") pod \"e2e-api-b-3689d4-openshift-default-8649d97477-4d497\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.497809 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.497721 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:58.521476 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.521395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:58.614588 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.614550 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:58.657661 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:37:58.657631 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3786c50_d023_4b42_be71_f42a8f9e3119.slice/crio-56e107ecf5ac9b61be34f806aac4d8b92c1af95d57f4d0a4f7a534290a780eb3 WatchSource:0}: Error finding container 56e107ecf5ac9b61be34f806aac4d8b92c1af95d57f4d0a4f7a534290a780eb3: Status 404 returned error can't find the container with id 56e107ecf5ac9b61be34f806aac4d8b92c1af95d57f4d0a4f7a534290a780eb3 Jul 08 18:37:58.659260 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.659237 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497"] Jul 08 18:37:58.660079 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.660036 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:37:58.660181 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.660110 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:37:58.660181 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.660153 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:37:58.715259 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:58.715225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:37:59.012707 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:59.012667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" event={"ID":"a3786c50-d023-4b42-be71-f42a8f9e3119","Type":"ContainerStarted","Data":"e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd"} Jul 08 18:37:59.012890 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:59.012726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" event={"ID":"a3786c50-d023-4b42-be71-f42a8f9e3119","Type":"ContainerStarted","Data":"56e107ecf5ac9b61be34f806aac4d8b92c1af95d57f4d0a4f7a534290a780eb3"} Jul 08 18:37:59.033273 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:59.033231 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" podStartSLOduration=1.033215213 podStartE2EDuration="1.033215213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:37:59.031774908 +0000 UTC m=+2034.716981677" watchObservedRunningTime="2026-07-08 18:37:59.033215213 +0000 UTC m=+2034.718421982" Jul 08 18:37:59.498165 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:59.498132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:37:59.502682 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:37:59.502661 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:00.017886 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:00.017859 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:00.018731 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:00.018710 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:01.005343 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:01.005293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:01.086340 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:01.086299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:01.162623 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:01.162500 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:01.253668 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:01.253630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:01.565990 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:01.565960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:01.666298 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:01.666262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:05.508848 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:05.508794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:08.011006 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:08.010966 2580 patch_prober.go:28] interesting pod/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:38:08.011421 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:08.011032 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:38:09.081287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:09.081256 2580 generic.go:356] "Generic (PLEG): container finished" podID="608be071-c1c2-4434-bd9e-353dd96db76e" containerID="feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011" exitCode=0 Jul 08 18:38:09.081735 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:09.081328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" event={"ID":"608be071-c1c2-4434-bd9e-353dd96db76e","Type":"ContainerDied","Data":"feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011"} Jul 08 18:38:09.081799 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:09.081752 2580 scope.go:117] "RemoveContainer" containerID="feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011" Jul 08 18:38:09.082844 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:09.082802 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:09.082961 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:09.082888 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:09.082961 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:09.082923 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:10.088367 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:10.088331 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" event={"ID":"608be071-c1c2-4434-bd9e-353dd96db76e","Type":"ContainerStarted","Data":"bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623"} Jul 08 18:38:10.929620 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:10.929590 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:38:10.929808 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:10.929788 2580 patch_prober.go:28] interesting pod/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 18:38:10.929894 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:10.929865 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 18:38:11.930247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:11.930209 2580 patch_prober.go:28] interesting pod/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 18:38:11.930651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:11.930264 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 18:38:12.930363 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:12.930326 2580 patch_prober.go:28] interesting pod/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 18:38:12.930785 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:12.930385 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 18:38:13.812123 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:13.812094 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:13.930263 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:13.930221 2580 patch_prober.go:28] interesting pod/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 18:38:13.930446 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:13.930281 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 18:38:14.267721 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:14.267681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:14.367774 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:14.367683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:14.467870 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:14.467823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:14.625448 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:14.625371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:14.934072 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:14.934035 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:38:14.934607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:14.934585 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:38:14.935329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:14.935308 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:38:15.486686 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:15.485613 2580 patch_prober.go:28] interesting pod/e2e-api-b-3689d4-openshift-default-8649d97477-4d497 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:34378->10.132.0.62:15021: read: connection reset by peer" start-of-body= Jul 08 18:38:15.486686 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:15.485679 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:34378->10.132.0.62:15021: read: connection reset by peer" Jul 08 18:38:16.121139 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:16.121107 2580 generic.go:356] "Generic (PLEG): container finished" podID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerID="e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd" exitCode=0 Jul 08 18:38:16.121530 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:16.121173 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" event={"ID":"a3786c50-d023-4b42-be71-f42a8f9e3119","Type":"ContainerDied","Data":"e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd"} Jul 08 18:38:16.121611 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:16.121596 2580 scope.go:117] "RemoveContainer" containerID="e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd" Jul 08 18:38:16.122659 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:16.122632 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:16.122722 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:16.122710 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:16.122770 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:16.122755 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:17.127730 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:17.127699 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" event={"ID":"a3786c50-d023-4b42-be71-f42a8f9e3119","Type":"ContainerStarted","Data":"bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b"} Jul 08 18:38:17.498005 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:17.497968 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:17.498234 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:17.498212 2580 patch_prober.go:28] interesting pod/e2e-api-b-3689d4-openshift-default-8649d97477-4d497 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 18:38:17.498281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:17.498265 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 18:38:18.498327 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:18.498296 2580 patch_prober.go:28] interesting pod/e2e-api-b-3689d4-openshift-default-8649d97477-4d497 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 18:38:18.498709 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:18.498362 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 18:38:19.498858 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:19.498806 2580 patch_prober.go:28] interesting pod/e2e-api-b-3689d4-openshift-default-8649d97477-4d497 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 18:38:19.499261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:19.498884 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 18:38:20.385382 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:20.385344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:20.449641 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:20.449611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:20.498456 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:20.498426 2580 patch_prober.go:28] interesting pod/e2e-api-b-3689d4-openshift-default-8649d97477-4d497 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 18:38:20.498643 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:20.498485 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 18:38:20.669521 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:20.669482 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br"] Jul 08 18:38:20.670011 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:20.669685 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" containerID="cri-o://bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623" gracePeriod=30 Jul 08 18:38:20.697828 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:20.697778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:21.059498 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.059400 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:21.453303 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.453271 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:21.502161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.502137 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:21.502559 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.502538 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:21.503191 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.503172 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:21.527950 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.527919 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:21.591127 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.591088 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:21.758667 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:21.758591 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:25.932442 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.932384 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:38:25.999852 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.999806 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk9v9\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-kube-api-access-bk9v9\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.999863 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-data\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.999898 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-certs\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.999921 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608be071-c1c2-4434-bd9e-353dd96db76e-istiod-ca-cert\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.999941 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-istio-token\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.999966 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608be071-c1c2-4434-bd9e-353dd96db76e-istio-podinfo\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000009 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:25.999996 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-credential-socket\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000333 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000046 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-envoy\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000333 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000245 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:26.000439 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000324 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:26.000439 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000339 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/608be071-c1c2-4434-bd9e-353dd96db76e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:38:26.000439 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000395 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-socket\") pod \"608be071-c1c2-4434-bd9e-353dd96db76e\" (UID: \"608be071-c1c2-4434-bd9e-353dd96db76e\") " Jul 08 18:38:26.000679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000652 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:26.000790 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000736 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-data" (OuterVolumeSpecName: "istio-data") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:26.000883 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000865 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.000934 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000891 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.000934 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000908 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.000934 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000920 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608be071-c1c2-4434-bd9e-353dd96db76e-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.001045 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.000932 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.002126 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.002094 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-kube-api-access-bk9v9" (OuterVolumeSpecName: "kube-api-access-bk9v9") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "kube-api-access-bk9v9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:38:26.002236 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.002163 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/608be071-c1c2-4434-bd9e-353dd96db76e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:38:26.002506 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.002487 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-istio-token" (OuterVolumeSpecName: "istio-token") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:38:26.002554 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.002528 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "608be071-c1c2-4434-bd9e-353dd96db76e" (UID: "608be071-c1c2-4434-bd9e-353dd96db76e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:26.101755 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.101729 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.101755 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.101751 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608be071-c1c2-4434-bd9e-353dd96db76e-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.101755 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.101761 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608be071-c1c2-4434-bd9e-353dd96db76e-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.101979 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.101769 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk9v9\" (UniqueName: \"kubernetes.io/projected/608be071-c1c2-4434-bd9e-353dd96db76e-kube-api-access-bk9v9\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:26.182593 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.182495 2580 generic.go:356] "Generic (PLEG): container finished" podID="608be071-c1c2-4434-bd9e-353dd96db76e" containerID="bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623" exitCode=0 Jul 08 18:38:26.182593 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.182583 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" event={"ID":"608be071-c1c2-4434-bd9e-353dd96db76e","Type":"ContainerDied","Data":"bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623"} Jul 08 18:38:26.182873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.182670 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" event={"ID":"608be071-c1c2-4434-bd9e-353dd96db76e","Type":"ContainerDied","Data":"9b1d83a0c1c35eb9c07bfb43cf71058cb72c41787b8156692f83ec74c00e9f39"} Jul 08 18:38:26.182873 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.182707 2580 scope.go:117] "RemoveContainer" containerID="bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623" Jul 08 18:38:26.183367 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.182927 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br" Jul 08 18:38:26.193183 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.193167 2580 scope.go:117] "RemoveContainer" containerID="feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011" Jul 08 18:38:26.201928 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.201908 2580 scope.go:117] "RemoveContainer" containerID="bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623" Jul 08 18:38:26.202208 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:38:26.202177 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623\": container with ID starting with bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623 not found: ID does not exist" containerID="bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623" Jul 08 18:38:26.202317 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.202215 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623"} err="failed to get container status \"bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623\": rpc error: code = NotFound desc = could not find container \"bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623\": container with ID starting with bd391996c1ff0c7eb6b57b516f64e1a4d4fba988d5a7cb0c3e4ba35f9e1e7623 not found: ID does not exist" Jul 08 18:38:26.202317 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.202238 2580 scope.go:117] "RemoveContainer" containerID="feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011" Jul 08 18:38:26.202513 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:38:26.202496 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011\": container with ID starting with feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011 not found: ID does not exist" containerID="feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011" Jul 08 18:38:26.202577 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.202517 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011"} err="failed to get container status \"feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011\": rpc error: code = NotFound desc = could not find container \"feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011\": container with ID starting with feff8ab31206c42178cc4956b0c1693b9c20cdc1b98042ddd810982d36fd1011 not found: ID does not exist" Jul 08 18:38:26.206604 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.206582 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br"] Jul 08 18:38:26.212315 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.212296 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-34c6f7-openshift-default-5d665d595c-zx4br"] Jul 08 18:38:26.870785 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:26.870743 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" path="/var/lib/kubelet/pods/608be071-c1c2-4434-bd9e-353dd96db76e/volumes" Jul 08 18:38:27.651801 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:27.651762 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:27.755858 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:27.755811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:27.903531 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:27.903453 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497"] Jul 08 18:38:27.903732 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:27.903707 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" containerID="cri-o://bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b" gracePeriod=30 Jul 08 18:38:27.920161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:27.920136 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:28.496573 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.496540 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:28.770939 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.770795 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27"] Jul 08 18:38:28.771527 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.771502 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" Jul 08 18:38:28.771706 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.771686 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" Jul 08 18:38:28.771820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.771722 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" Jul 08 18:38:28.771820 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.771733 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8231cd0f-551c-4ca9-bc91-40b5ecc3c1b0" containerName="istio-proxy" Jul 08 18:38:28.771969 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.771884 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" Jul 08 18:38:28.771969 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.771895 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" Jul 08 18:38:28.776618 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.776600 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.779247 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.779229 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-deca3c-openshift-default-dockercfg-rdtcw\"" Jul 08 18:38:28.787164 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.787140 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27"] Jul 08 18:38:28.796077 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.796035 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:28.826380 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826356 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-envoy\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826495 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-certs\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826495 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-podinfo\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826626 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-credential-socket\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826626 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826576 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-data\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826741 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istiod-ca-cert\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826741 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-token\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826868 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlv6j\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-kube-api-access-zlv6j\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.826868 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.826762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-socket\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.927513 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927484 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-envoy\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.927696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-certs\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.927696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-podinfo\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.927696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-credential-socket\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.927696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-data\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.927696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istiod-ca-cert\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928028 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-token\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928028 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlv6j\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-kube-api-access-zlv6j\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928028 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.927779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-socket\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928186 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.928032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-credential-socket\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928186 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.928172 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-data\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928425 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.928397 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-socket\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928506 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.928430 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-certs\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.928506 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.928434 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istiod-ca-cert\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.930412 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.930395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-envoy\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.930541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.930525 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-podinfo\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.938541 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.938512 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-token\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:28.938960 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:28.938937 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlv6j\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-kube-api-access-zlv6j\") pod \"e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:29.088174 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:29.088092 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:29.233366 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:29.233339 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27"] Jul 08 18:38:29.234137 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:38:29.234115 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7941b5_c2a8_428c_b557_c5d7cb8910c1.slice/crio-3583603dec90be8ea865b01d7a1351035805188af00dc29308ff8839346ba441 WatchSource:0}: Error finding container 3583603dec90be8ea865b01d7a1351035805188af00dc29308ff8839346ba441: Status 404 returned error can't find the container with id 3583603dec90be8ea865b01d7a1351035805188af00dc29308ff8839346ba441 Jul 08 18:38:29.236692 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:29.236659 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:29.236792 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:29.236738 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:29.236792 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:29.236778 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:29.892026 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:29.891994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:30.207026 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:30.206946 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" event={"ID":"0a7941b5-c2a8-428c-b557-c5d7cb8910c1","Type":"ContainerStarted","Data":"99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41"} Jul 08 18:38:30.207026 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:30.207002 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" event={"ID":"0a7941b5-c2a8-428c-b557-c5d7cb8910c1","Type":"ContainerStarted","Data":"3583603dec90be8ea865b01d7a1351035805188af00dc29308ff8839346ba441"} Jul 08 18:38:30.229589 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:30.229541 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" podStartSLOduration=2.229528735 podStartE2EDuration="2.229528735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:38:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:38:30.227745884 +0000 UTC m=+2065.912952654" watchObservedRunningTime="2026-07-08 18:38:30.229528735 +0000 UTC m=+2065.914735503" Jul 08 18:38:31.089273 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:31.089240 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:31.094335 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:31.094311 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:31.212900 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:31.212872 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:31.213898 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:31.213868 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:32.147078 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:32.147040 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:32.332354 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:32.332319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:32.432363 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:32.432281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:32.618169 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:32.618135 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:32.886364 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:32.886281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:32.986628 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:32.986601 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:33.169429 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.169410 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:33.227518 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.227485 2580 generic.go:356] "Generic (PLEG): container finished" podID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerID="bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b" exitCode=0 Jul 08 18:38:33.227651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.227535 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" Jul 08 18:38:33.227651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.227572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" event={"ID":"a3786c50-d023-4b42-be71-f42a8f9e3119","Type":"ContainerDied","Data":"bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b"} Jul 08 18:38:33.227651 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.227633 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497" event={"ID":"a3786c50-d023-4b42-be71-f42a8f9e3119","Type":"ContainerDied","Data":"56e107ecf5ac9b61be34f806aac4d8b92c1af95d57f4d0a4f7a534290a780eb3"} Jul 08 18:38:33.227767 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.227667 2580 scope.go:117] "RemoveContainer" containerID="bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b" Jul 08 18:38:33.236864 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.236846 2580 scope.go:117] "RemoveContainer" containerID="e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd" Jul 08 18:38:33.244792 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.244776 2580 scope.go:117] "RemoveContainer" containerID="bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b" Jul 08 18:38:33.245040 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:38:33.245022 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b\": container with ID starting with bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b not found: ID does not exist" containerID="bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b" Jul 08 18:38:33.245124 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.245054 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b"} err="failed to get container status \"bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b\": rpc error: code = NotFound desc = could not find container \"bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b\": container with ID starting with bb48cb31ebf847c6dd46b3d9a0bb7fcc0f5256773cbd281936618ed74ed7972b not found: ID does not exist" Jul 08 18:38:33.245124 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.245077 2580 scope.go:117] "RemoveContainer" containerID="e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd" Jul 08 18:38:33.245318 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:38:33.245303 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd\": container with ID starting with e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd not found: ID does not exist" containerID="e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd" Jul 08 18:38:33.245362 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.245324 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd"} err="failed to get container status \"e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd\": rpc error: code = NotFound desc = could not find container \"e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd\": container with ID starting with e31d077c412a72ba0f95c3d89b1ed8012768a9fe9d893c2217578fdc09a3afdd not found: ID does not exist" Jul 08 18:38:33.266634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266611 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3786c50-d023-4b42-be71-f42a8f9e3119-istiod-ca-cert\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.266736 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266677 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-socket\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.266736 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266713 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-credential-socket\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.266874 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266751 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-certs\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.266874 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266777 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-podinfo\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.266874 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266808 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-envoy\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.266874 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266863 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsh5w\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-kube-api-access-tsh5w\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.267079 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266911 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-data\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.267079 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.266942 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-token\") pod \"a3786c50-d023-4b42-be71-f42a8f9e3119\" (UID: \"a3786c50-d023-4b42-be71-f42a8f9e3119\") " Jul 08 18:38:33.267079 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267047 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:33.267244 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267100 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:33.267244 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267059 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3786c50-d023-4b42-be71-f42a8f9e3119-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:38:33.267385 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267365 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.267447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267389 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3786c50-d023-4b42-be71-f42a8f9e3119-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.267447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267402 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.267553 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267506 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:33.267920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.267893 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:33.269197 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.269173 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:38:33.269287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.269201 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:38:33.269287 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.269205 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-kube-api-access-tsh5w" (OuterVolumeSpecName: "kube-api-access-tsh5w") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "kube-api-access-tsh5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:38:33.269366 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.269330 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3786c50-d023-4b42-be71-f42a8f9e3119" (UID: "a3786c50-d023-4b42-be71-f42a8f9e3119"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:38:33.367998 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.367974 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.367998 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.367996 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.368173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.368006 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.368173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.368014 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsh5w\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-kube-api-access-tsh5w\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.368173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.368022 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.368173 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.368029 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3786c50-d023-4b42-be71-f42a8f9e3119-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:38:33.550123 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.550091 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497"] Jul 08 18:38:33.553768 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:33.553746 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-3689d4-openshift-default-8649d97477-4d497"] Jul 08 18:38:34.869719 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:34.869677 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" path="/var/lib/kubelet/pods/a3786c50-d023-4b42-be71-f42a8f9e3119/volumes" Jul 08 18:38:36.666117 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:36.666085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:37.010490 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.010398 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:37.315559 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.315480 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz"] Jul 08 18:38:37.316144 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316128 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" Jul 08 18:38:37.316203 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316149 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="608be071-c1c2-4434-bd9e-353dd96db76e" containerName="istio-proxy" Jul 08 18:38:37.316203 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316178 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" Jul 08 18:38:37.316203 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316186 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" Jul 08 18:38:37.316203 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316195 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" Jul 08 18:38:37.316203 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316203 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" Jul 08 18:38:37.316356 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316304 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" Jul 08 18:38:37.316356 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.316317 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3786c50-d023-4b42-be71-f42a8f9e3119" containerName="istio-proxy" Jul 08 18:38:37.321142 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.321125 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.324023 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.324004 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e6a5a1-openshift-default-dockercfg-2cgl6\"" Jul 08 18:38:37.330696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.330677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz"] Jul 08 18:38:37.351948 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.351915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:37.404100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-token\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpqcq\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-kube-api-access-kpqcq\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404156 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-podinfo\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-certs\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404254 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404238 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9401966-35f1-4b34-8a3e-9d3204b62d31-istiod-ca-cert\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-data\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-credential-socket\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-socket\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.404478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.404438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-envoy\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505021 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.504992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-token\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505021 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpqcq\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-kube-api-access-kpqcq\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-podinfo\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-certs\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9401966-35f1-4b34-8a3e-9d3204b62d31-istiod-ca-cert\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-data\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-credential-socket\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505281 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-socket\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-envoy\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505664 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505616 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-certs\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.505722 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.505676 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-data\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.506097 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.506058 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-socket\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.506253 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.506138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9401966-35f1-4b34-8a3e-9d3204b62d31-istiod-ca-cert\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.506329 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.506258 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-credential-socket\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.507609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.507581 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-envoy\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.507757 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.507621 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-podinfo\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.514380 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.514360 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpqcq\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-kube-api-access-kpqcq\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.514530 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.514511 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-token\") pod \"e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.632668 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.632563 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:37.718858 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.717155 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:37.852857 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.844732 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz"] Jul 08 18:38:37.860019 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:38:37.859989 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9401966_35f1_4b34_8a3e_9d3204b62d31.slice/crio-fb4961b8fcec440094d6f50de935df38ea77b484808948805f8d706075fe3000 WatchSource:0}: Error finding container fb4961b8fcec440094d6f50de935df38ea77b484808948805f8d706075fe3000: Status 404 returned error can't find the container with id fb4961b8fcec440094d6f50de935df38ea77b484808948805f8d706075fe3000 Jul 08 18:38:37.862107 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.862073 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:37.862197 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.862147 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:37.862197 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:37.862180 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:38.258853 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:38.258728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" event={"ID":"d9401966-35f1-4b34-8a3e-9d3204b62d31","Type":"ContainerStarted","Data":"9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d"} Jul 08 18:38:38.258853 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:38.258801 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" event={"ID":"d9401966-35f1-4b34-8a3e-9d3204b62d31","Type":"ContainerStarted","Data":"fb4961b8fcec440094d6f50de935df38ea77b484808948805f8d706075fe3000"} Jul 08 18:38:38.288750 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:38.288706 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" podStartSLOduration=1.288691878 podStartE2EDuration="1.288691878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:38:38.286339294 +0000 UTC m=+2073.971546062" watchObservedRunningTime="2026-07-08 18:38:38.288691878 +0000 UTC m=+2073.973898648" Jul 08 18:38:38.633487 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:38.633391 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:38.648942 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:38.648911 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:39.263993 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:39.263962 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:39.265029 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:39.265011 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:40.259064 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:40.258929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:40.356470 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:40.356428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:40.424467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:40.424428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:40.590654 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:40.590558 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:41.029861 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:41.029811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:41.127265 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:41.127233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:44.761152 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:44.761056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:45.299382 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:45.299350 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerID="99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41" exitCode=0 Jul 08 18:38:45.299551 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:45.299424 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" event={"ID":"0a7941b5-c2a8-428c-b557-c5d7cb8910c1","Type":"ContainerDied","Data":"99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41"} Jul 08 18:38:45.299824 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:45.299811 2580 scope.go:117] "RemoveContainer" containerID="99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41" Jul 08 18:38:45.300757 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:45.300731 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:45.300815 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:45.300797 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:45.300874 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:45.300829 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:46.213504 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:46.213471 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:46.306214 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:46.306179 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" event={"ID":"0a7941b5-c2a8-428c-b557-c5d7cb8910c1","Type":"ContainerStarted","Data":"13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7"} Jul 08 18:38:47.088943 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:47.088908 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:47.090250 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:47.090221 2580 patch_prober.go:28] interesting pod/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 18:38:47.090395 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:47.090300 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 18:38:48.088896 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:48.088829 2580 patch_prober.go:28] interesting pod/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 18:38:48.089277 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:48.088920 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 18:38:49.088763 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:49.088729 2580 patch_prober.go:28] interesting pod/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 18:38:49.088966 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:49.088782 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 18:38:50.089379 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:50.089342 2580 patch_prober.go:28] interesting pod/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 18:38:50.089786 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:50.089405 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 18:38:51.092309 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:51.092284 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:51.092708 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:51.092544 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:51.093315 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:51.093296 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:38:54.404208 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:54.404176 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:54.501480 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:54.501446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:55.305989 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:55.305945 2580 patch_prober.go:28] interesting pod/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 18:38:55.306200 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:55.306009 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 18:38:56.364932 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:56.364901 2580 generic.go:356] "Generic (PLEG): container finished" podID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerID="9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d" exitCode=0 Jul 08 18:38:56.365349 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:56.364975 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" event={"ID":"d9401966-35f1-4b34-8a3e-9d3204b62d31","Type":"ContainerDied","Data":"9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d"} Jul 08 18:38:56.365398 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:56.365352 2580 scope.go:117] "RemoveContainer" containerID="9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d" Jul 08 18:38:56.366318 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:56.366287 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:56.366411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:56.366353 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:56.366411 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:56.366385 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 18:38:57.371802 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:57.371768 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" event={"ID":"d9401966-35f1-4b34-8a3e-9d3204b62d31","Type":"ContainerStarted","Data":"919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72"} Jul 08 18:38:57.633489 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:57.633397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:38:57.633876 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:57.633748 2580 patch_prober.go:28] interesting pod/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 08 18:38:57.634084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:57.634058 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 08 18:38:58.199098 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:58.199062 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:38:58.633125 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:58.633032 2580 patch_prober.go:28] interesting pod/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 08 18:38:58.633727 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:58.633693 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 08 18:38:59.633463 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:59.633429 2580 patch_prober.go:28] interesting pod/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 08 18:38:59.633961 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:38:59.633492 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 08 18:39:00.633311 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:00.633274 2580 patch_prober.go:28] interesting pod/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 08 18:39:00.633618 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:00.633584 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 08 18:39:01.636574 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:01.636536 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:39:01.636957 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:01.636908 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:39:01.637548 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:01.637533 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:39:01.913349 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:01.913314 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:39:02.014027 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:02.013995 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:39:04.909675 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:04.909650 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:39:04.914788 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:04.914758 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:39:17.080439 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:17.080407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:39:17.146061 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:39:17.146032 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:49.909706 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:49.909626 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:50.294523 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:50.294432 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:50.393294 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:50.393263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:50.494164 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:50.494130 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:50.597960 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:50.597875 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:56.661605 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:56.661572 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:56.718963 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:56.718930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:56.907282 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:56.907252 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27"] Jul 08 18:41:56.907469 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:56.907450 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" containerID="cri-o://13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7" gracePeriod=30 Jul 08 18:41:56.923568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:56.923503 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:57.325894 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:57.325784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:57.746120 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:57.746086 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:57.846001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:57.845973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:41:58.017381 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:41:58.017297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:02.166649 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.166628 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:42:02.315087 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315004 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-data\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315087 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315052 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlv6j\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-kube-api-access-zlv6j\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315087 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315072 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-certs\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315096 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-envoy\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315127 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-podinfo\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315147 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-token\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315177 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istiod-ca-cert\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315218 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-socket\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315467 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315273 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-credential-socket\") pod \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\" (UID: \"0a7941b5-c2a8-428c-b557-c5d7cb8910c1\") " Jul 08 18:42:02.315779 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315747 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:42:02.315940 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315874 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:02.315940 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315585 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:02.315940 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.315918 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:02.316156 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.316047 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:02.317813 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.317782 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:02.317813 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.317796 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-kube-api-access-zlv6j" (OuterVolumeSpecName: "kube-api-access-zlv6j") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "kube-api-access-zlv6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:42:02.318001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.317920 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerID="13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7" exitCode=0 Jul 08 18:42:02.318001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.317968 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:42:02.318001 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.317979 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" Jul 08 18:42:02.318142 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.318000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" event={"ID":"0a7941b5-c2a8-428c-b557-c5d7cb8910c1","Type":"ContainerDied","Data":"13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7"} Jul 08 18:42:02.318142 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.318058 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27" event={"ID":"0a7941b5-c2a8-428c-b557-c5d7cb8910c1","Type":"ContainerDied","Data":"3583603dec90be8ea865b01d7a1351035805188af00dc29308ff8839346ba441"} Jul 08 18:42:02.318142 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.318088 2580 scope.go:117] "RemoveContainer" containerID="13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7" Jul 08 18:42:02.318261 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.318154 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a7941b5-c2a8-428c-b557-c5d7cb8910c1" (UID: "0a7941b5-c2a8-428c-b557-c5d7cb8910c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:42:02.337819 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.337800 2580 scope.go:117] "RemoveContainer" containerID="99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41" Jul 08 18:42:02.346095 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.346079 2580 scope.go:117] "RemoveContainer" containerID="13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7" Jul 08 18:42:02.346354 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:42:02.346334 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7\": container with ID starting with 13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7 not found: ID does not exist" containerID="13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7" Jul 08 18:42:02.346434 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.346367 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7"} err="failed to get container status \"13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7\": rpc error: code = NotFound desc = could not find container \"13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7\": container with ID starting with 13833fa269e41a5193ee1f3155f9fa6b5016abbaa399f97f310a90bfcf2444b7 not found: ID does not exist" Jul 08 18:42:02.346434 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.346392 2580 scope.go:117] "RemoveContainer" containerID="99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41" Jul 08 18:42:02.346658 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:42:02.346641 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41\": container with ID starting with 99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41 not found: ID does not exist" containerID="99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41" Jul 08 18:42:02.346703 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.346664 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41"} err="failed to get container status \"99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41\": rpc error: code = NotFound desc = could not find container \"99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41\": container with ID starting with 99d410bc071a2cfb54bafc2869dfef840625089312aabb25d2031d7de8dd1e41 not found: ID does not exist" Jul 08 18:42:02.416948 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416916 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlv6j\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-kube-api-access-zlv6j\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.416948 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416942 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.416948 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416951 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.417190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416959 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.417190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416967 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.417190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416975 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.417190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416983 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.417190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416990 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.417190 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.416997 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a7941b5-c2a8-428c-b557-c5d7cb8910c1-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:02.642057 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.642031 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27"] Jul 08 18:42:02.646163 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.646140 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-deca3c-openshift-default-cbb96b7ff-fww27"] Jul 08 18:42:02.870510 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:02.870477 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" path="/var/lib/kubelet/pods/0a7941b5-c2a8-428c-b557-c5d7cb8910c1/volumes" Jul 08 18:42:03.773131 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:03.773097 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:03.870338 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:03.870311 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:04.045580 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:04.045506 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz"] Jul 08 18:42:04.045732 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:04.045717 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" containerID="cri-o://919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72" gracePeriod=30 Jul 08 18:42:04.052648 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:04.052600 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:04.851761 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:04.851731 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85dc8bd77-cwbgl_9f697f0a-416e-4cc8-baec-e7fe8d2e45ca/authorino/0.log" Jul 08 18:42:09.305569 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.305547 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:42:09.356114 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.356043 2580 generic.go:356] "Generic (PLEG): container finished" podID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerID="919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72" exitCode=0 Jul 08 18:42:09.356114 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.356100 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" Jul 08 18:42:09.356293 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.356128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" event={"ID":"d9401966-35f1-4b34-8a3e-9d3204b62d31","Type":"ContainerDied","Data":"919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72"} Jul 08 18:42:09.356293 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.356184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz" event={"ID":"d9401966-35f1-4b34-8a3e-9d3204b62d31","Type":"ContainerDied","Data":"fb4961b8fcec440094d6f50de935df38ea77b484808948805f8d706075fe3000"} Jul 08 18:42:09.356293 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.356207 2580 scope.go:117] "RemoveContainer" containerID="919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72" Jul 08 18:42:09.364829 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.364810 2580 scope.go:117] "RemoveContainer" containerID="9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d" Jul 08 18:42:09.372822 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.372803 2580 scope.go:117] "RemoveContainer" containerID="919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72" Jul 08 18:42:09.373109 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:42:09.373091 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72\": container with ID starting with 919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72 not found: ID does not exist" containerID="919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72" Jul 08 18:42:09.373172 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.373117 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72"} err="failed to get container status \"919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72\": rpc error: code = NotFound desc = could not find container \"919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72\": container with ID starting with 919199ce192ba7da34a92b694f0cf5bf66935a3e0a2fd50a31aed2e8671afb72 not found: ID does not exist" Jul 08 18:42:09.373172 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.373134 2580 scope.go:117] "RemoveContainer" containerID="9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d" Jul 08 18:42:09.373384 ip-10-0-141-56 kubenswrapper[2580]: E0708 18:42:09.373364 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d\": container with ID starting with 9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d not found: ID does not exist" containerID="9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d" Jul 08 18:42:09.373439 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.373394 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d"} err="failed to get container status \"9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d\": rpc error: code = NotFound desc = could not find container \"9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d\": container with ID starting with 9749f74c434eb7e05bed23454a2d02a94754a2258ecbac8cbe1252c7b32a911d not found: ID does not exist" Jul 08 18:42:09.377709 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377688 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-socket\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.377802 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377729 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-token\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.377802 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377756 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpqcq\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-kube-api-access-kpqcq\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.377941 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377888 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-podinfo\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.377941 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377924 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-data\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.378048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377936 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:09.378048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377951 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-credential-socket\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.378048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.377986 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9401966-35f1-4b34-8a3e-9d3204b62d31-istiod-ca-cert\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.378048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378028 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-envoy\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.378244 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378120 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-certs\") pod \"d9401966-35f1-4b34-8a3e-9d3204b62d31\" (UID: \"d9401966-35f1-4b34-8a3e-9d3204b62d31\") " Jul 08 18:42:09.378347 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378306 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:09.378568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378407 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9401966-35f1-4b34-8a3e-9d3204b62d31-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:42:09.378568 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378546 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:09.378784 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378696 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:09.378905 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378782 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-credential-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.378905 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378803 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9401966-35f1-4b34-8a3e-9d3204b62d31-istiod-ca-cert\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.378905 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.378818 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-socket\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.380373 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.380345 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-kube-api-access-kpqcq" (OuterVolumeSpecName: "kube-api-access-kpqcq") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "kube-api-access-kpqcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:42:09.380570 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.380547 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:42:09.380758 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.380737 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:42:09.380885 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.380787 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9401966-35f1-4b34-8a3e-9d3204b62d31" (UID: "d9401966-35f1-4b34-8a3e-9d3204b62d31"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:42:09.459197 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.459175 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-r6vjm_2fddb3ef-91b0-4934-b9c5-838e4324b48e/manager/0.log" Jul 08 18:42:09.479677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.479653 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-podinfo\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.479677 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.479674 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-data\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.479816 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.479683 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-envoy\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.479816 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.479692 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9401966-35f1-4b34-8a3e-9d3204b62d31-workload-certs\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.479816 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.479700 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-istio-token\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.479816 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.479708 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpqcq\" (UniqueName: \"kubernetes.io/projected/d9401966-35f1-4b34-8a3e-9d3204b62d31-kube-api-access-kpqcq\") on node \"ip-10-0-141-56.ec2.internal\" DevicePath \"\"" Jul 08 18:42:09.680403 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.680379 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz"] Jul 08 18:42:09.684606 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.684585 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e6a5a1-openshift-default-64cd6bdc4c-4r7rz"] Jul 08 18:42:09.712676 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.712656 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-598cd5774c-ssxx4_33add953-1c37-468e-bd2f-9c112d51bcb1/manager/0.log" Jul 08 18:42:09.826360 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.826343 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r662d_67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5/server/0.log" Jul 08 18:42:09.942166 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:09.942099 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-smjqr_00222fa8-23cb-4a82-bdfb-154cb3663b3e/manager/0.log" Jul 08 18:42:10.324701 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:10.324675 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-z59wh_8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f/copy-manifests/0.log" Jul 08 18:42:10.332408 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:10.332389 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-z59wh_8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f/manager/0.log" Jul 08 18:42:10.869218 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:10.869191 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" path="/var/lib/kubelet/pods/d9401966-35f1-4b34-8a3e-9d3204b62d31/volumes" Jul 08 18:42:11.613960 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:11.613917 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85dc8bd77-cwbgl_9f697f0a-416e-4cc8-baec-e7fe8d2e45ca/authorino/0.log" Jul 08 18:42:11.740469 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:11.740438 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wnblj_8d39a9bc-ec97-4066-93db-c55f9890c05a/manager/0.log" Jul 08 18:42:11.854210 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:11.854185 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mm2q4_e867ceca-ea46-45c0-9659-e81210d2a2de/manager/0.log" Jul 08 18:42:12.085025 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:12.084994 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7qrwt_ba17c761-c27a-4018-a030-1264f90809e6/registry-server/0.log" Jul 08 18:42:12.456153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:12.456100 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cr2rz_d6fdf493-475a-4184-8ad8-1077c89ecb38/manager/0.log" Jul 08 18:42:12.835823 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:12.835741 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wscxz_c6a37a64-1b14-4e6a-a15c-91ed60294d7f/istio-proxy/0.log" Jul 08 18:42:13.081101 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:13.081075 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-29tmb_5ba67f36-15a0-4cd4-9dd5-2376c770a2c9/kube-auth-proxy/0.log" Jul 08 18:42:13.319417 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:13.319389 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vfz68_59a21420-9066-4247-939c-aef787050364/istio-proxy/1.log" Jul 08 18:42:13.443819 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:13.443790 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-d8l8b_5d010009-a12f-424c-ae94-96042a1fedd2/payload-pre-processing/0.log" Jul 08 18:42:13.696717 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:13.696687 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d459b6969-8sg97_6c74951d-5706-4a46-9163-5b9d3dc67042/router/0.log" Jul 08 18:42:14.045048 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:14.044971 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-f2sxz_4166db4f-acea-4220-bc8f-0c8636758a5a/main/0.log" Jul 08 18:42:14.400390 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:14.400362 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5x8f2_8a9a32d9-5490-4845-8750-f7ba11245d95/main/0.log" Jul 08 18:42:18.480762 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:18.480733 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:18.576043 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:18.576013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:21.575696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:21.575668 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-znlq7_7ee9becb-c169-4398-b46e-7daa536542a4/global-pull-secret-syncer/0.log" Jul 08 18:42:21.630412 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:21.630379 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" Jul 08 18:42:21.782214 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:21.782188 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-56.ec2.internal_836915b6922e9a09409b13134d65a463/haproxy/0.log" Jul 08 18:42:26.470375 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:26.470346 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85dc8bd77-cwbgl_9f697f0a-416e-4cc8-baec-e7fe8d2e45ca/authorino/0.log" Jul 08 18:42:26.502291 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:26.502271 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wnblj_8d39a9bc-ec97-4066-93db-c55f9890c05a/manager/0.log" Jul 08 18:42:26.529924 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:26.529905 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mm2q4_e867ceca-ea46-45c0-9659-e81210d2a2de/manager/0.log" Jul 08 18:42:26.603194 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:26.603171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7qrwt_ba17c761-c27a-4018-a030-1264f90809e6/registry-server/0.log" Jul 08 18:42:26.889528 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:26.889506 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cr2rz_d6fdf493-475a-4184-8ad8-1077c89ecb38/manager/0.log" Jul 08 18:42:28.603703 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:28.603673 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-clrnw_3fddd715-cce6-4e4d-a611-adc89869fd3e/cluster-monitoring-operator/0.log" Jul 08 18:42:28.796766 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:28.796740 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2pl47_be2bf90e-0bbf-404d-a675-131be80a9405/node-exporter/0.log" Jul 08 18:42:28.826607 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:28.826577 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2pl47_be2bf90e-0bbf-404d-a675-131be80a9405/kube-rbac-proxy/0.log" Jul 08 18:42:28.857127 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:28.857074 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2pl47_be2bf90e-0bbf-404d-a675-131be80a9405/init-textfile/0.log" Jul 08 18:42:30.993216 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:30.993177 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-jq98v_76d17d44-d39f-4b04-9443-28a71704f3a8/networking-console-plugin/0.log" Jul 08 18:42:31.639125 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639088 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz"] Jul 08 18:42:31.639487 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639475 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" Jul 08 18:42:31.639533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639489 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" Jul 08 18:42:31.639533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639501 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" Jul 08 18:42:31.639533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639507 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" Jul 08 18:42:31.639533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639514 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" Jul 08 18:42:31.639533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639520 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" Jul 08 18:42:31.639533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639527 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" Jul 08 18:42:31.639533 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639532 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" Jul 08 18:42:31.639739 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639601 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7941b5-c2a8-428c-b557-c5d7cb8910c1" containerName="istio-proxy" Jul 08 18:42:31.639739 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639609 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" Jul 08 18:42:31.639739 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.639616 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9401966-35f1-4b34-8a3e-9d3204b62d31" containerName="istio-proxy" Jul 08 18:42:31.641224 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.641199 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/2.log" Jul 08 18:42:31.642956 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.642935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.645609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.645588 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9mvv9\"/\"openshift-service-ca.crt\"" Jul 08 18:42:31.645609 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.645606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9mvv9\"/\"default-dockercfg-tjq4t\"" Jul 08 18:42:31.645817 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.645648 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9mvv9\"/\"kube-root-ca.crt\"" Jul 08 18:42:31.646469 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.646444 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4vx47_6525a595-633b-454a-931a-5c2c55f5b8b4/console-operator/3.log" Jul 08 18:42:31.654035 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.654013 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz"] Jul 08 18:42:31.772733 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.772702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-podres\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.772920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.772769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-proc\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.772920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.772850 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-lib-modules\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.772920 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.772876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-sys\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.773027 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.772937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spjcx\" (UniqueName: \"kubernetes.io/projected/f7c3670c-54b2-412c-87de-fcf5aae6e87b-kube-api-access-spjcx\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.873976 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.873943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-proc\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.873984 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-lib-modules\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.874010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-sys\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.874066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spjcx\" (UniqueName: \"kubernetes.io/projected/f7c3670c-54b2-412c-87de-fcf5aae6e87b-kube-api-access-spjcx\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.874075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-proc\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.874131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-podres\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874153 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.874133 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-lib-modules\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874347 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.874157 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-sys\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.874347 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.874225 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7c3670c-54b2-412c-87de-fcf5aae6e87b-podres\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.882169 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.882149 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spjcx\" (UniqueName: \"kubernetes.io/projected/f7c3670c-54b2-412c-87de-fcf5aae6e87b-kube-api-access-spjcx\") pod \"perf-node-gather-daemonset-28gzz\" (UID: \"f7c3670c-54b2-412c-87de-fcf5aae6e87b\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:31.953792 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:31.953711 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:32.088317 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:32.088289 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz"] Jul 08 18:42:32.090024 ip-10-0-141-56 kubenswrapper[2580]: W0708 18:42:32.089996 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7c3670c_54b2_412c_87de_fcf5aae6e87b.slice/crio-39130439e6b672dfd5de2bd633c3bd99c6cbd533621619d0b28ddb2accca5f6d WatchSource:0}: Error finding container 39130439e6b672dfd5de2bd633c3bd99c6cbd533621619d0b28ddb2accca5f6d: Status 404 returned error can't find the container with id 39130439e6b672dfd5de2bd633c3bd99c6cbd533621619d0b28ddb2accca5f6d Jul 08 18:42:32.473380 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:32.473344 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" event={"ID":"f7c3670c-54b2-412c-87de-fcf5aae6e87b","Type":"ContainerStarted","Data":"bf6b26f49961fca341c8e8169e05fa99da943aff619c6d87e5702793c85be78d"} Jul 08 18:42:32.473556 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:32.473394 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" event={"ID":"f7c3670c-54b2-412c-87de-fcf5aae6e87b","Type":"ContainerStarted","Data":"39130439e6b672dfd5de2bd633c3bd99c6cbd533621619d0b28ddb2accca5f6d"} Jul 08 18:42:32.473556 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:32.473482 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:32.491365 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:32.491317 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" podStartSLOduration=1.491301506 podStartE2EDuration="1.491301506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:42:32.489185237 +0000 UTC m=+2308.174392008" watchObservedRunningTime="2026-07-08 18:42:32.491301506 +0000 UTC m=+2308.176508275" Jul 08 18:42:32.814809 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:32.814706 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-pnrdf_f0f85e84-3171-495f-a6f2-3d9def5ccc1c/volume-data-source-validator/0.log" Jul 08 18:42:33.891688 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:33.891663 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tj2cs_084a77c4-cf51-49c0-8cb0-1963bec63658/dns/0.log" Jul 08 18:42:33.913324 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:33.913301 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tj2cs_084a77c4-cf51-49c0-8cb0-1963bec63658/kube-rbac-proxy/0.log" Jul 08 18:42:33.985924 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:33.985899 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lngwj_6d5114bf-8ccf-4e07-8295-8e0d447ca9e5/dns-node-resolver/0.log" Jul 08 18:42:34.631732 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:34.631701 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tw7m_5e04054b-06b5-4b90-9891-3e66ab6942c0/node-ca/0.log" Jul 08 18:42:35.602567 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:35.602541 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wscxz_c6a37a64-1b14-4e6a-a15c-91ed60294d7f/istio-proxy/0.log" Jul 08 18:42:35.821672 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:35.821646 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-29tmb_5ba67f36-15a0-4cd4-9dd5-2376c770a2c9/kube-auth-proxy/0.log" Jul 08 18:42:35.874930 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:35.874862 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vfz68_59a21420-9066-4247-939c-aef787050364/istio-proxy/0.log" Jul 08 18:42:35.900229 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:35.900193 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vfz68_59a21420-9066-4247-939c-aef787050364/istio-proxy/1.log" Jul 08 18:42:35.938521 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:35.938496 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-d8l8b_5d010009-a12f-424c-ae94-96042a1fedd2/payload-pre-processing/0.log" Jul 08 18:42:36.039618 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:36.039594 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d459b6969-8sg97_6c74951d-5706-4a46-9163-5b9d3dc67042/router/0.log" Jul 08 18:42:36.768100 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:36.768072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xsrcw_3c398807-d5ee-4ab3-9cf6-94364606b1d6/serve-healthcheck-canary/0.log" Jul 08 18:42:37.221816 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:37.221784 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-lnv6h_4f7d7b3b-c770-4330-ba51-218d3b049f6b/insights-operator/0.log" Jul 08 18:42:37.222442 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:37.222425 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-lnv6h_4f7d7b3b-c770-4330-ba51-218d3b049f6b/insights-operator/1.log" Jul 08 18:42:37.244484 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:37.244463 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hndg9_bb83970e-f964-4be9-8882-4d1a8b25d3fa/kube-rbac-proxy/0.log" Jul 08 18:42:37.268526 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:37.268505 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hndg9_bb83970e-f964-4be9-8882-4d1a8b25d3fa/exporter/0.log" Jul 08 18:42:37.292679 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:37.292658 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hndg9_bb83970e-f964-4be9-8882-4d1a8b25d3fa/extractor/0.log" Jul 08 18:42:38.488849 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:38.488812 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-28gzz" Jul 08 18:42:39.535065 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:39.535038 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-r6vjm_2fddb3ef-91b0-4934-b9c5-838e4324b48e/manager/0.log" Jul 08 18:42:39.723849 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:39.723820 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-598cd5774c-ssxx4_33add953-1c37-468e-bd2f-9c112d51bcb1/manager/0.log" Jul 08 18:42:39.746781 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:39.746760 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r662d_67e3b6a2-a0bb-4d15-a585-5b6fac75c7e5/server/0.log" Jul 08 18:42:39.820915 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:39.820853 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-smjqr_00222fa8-23cb-4a82-bdfb-154cb3663b3e/manager/0.log" Jul 08 18:42:40.032447 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:40.032422 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-z59wh_8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f/manager/0.log" Jul 08 18:42:40.055646 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:40.055623 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-z59wh_8dbc1eaa-8eaa-4950-81e4-72aee32a7f7f/copy-manifests/0.log" Jul 08 18:42:41.886948 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:41.886916 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wscxz_c6a37a64-1b14-4e6a-a15c-91ed60294d7f/istio-proxy/0.log" Jul 08 18:42:42.070161 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:42.070085 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-29tmb_5ba67f36-15a0-4cd4-9dd5-2376c770a2c9/kube-auth-proxy/0.log" Jul 08 18:42:42.112084 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:42.112060 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vfz68_59a21420-9066-4247-939c-aef787050364/istio-proxy/0.log" Jul 08 18:42:42.138612 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:42.138591 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vfz68_59a21420-9066-4247-939c-aef787050364/istio-proxy/1.log" Jul 08 18:42:42.165645 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:42.165623 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-d8l8b_5d010009-a12f-424c-ae94-96042a1fedd2/payload-pre-processing/0.log" Jul 08 18:42:42.237209 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:42.237183 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d459b6969-8sg97_6c74951d-5706-4a46-9163-5b9d3dc67042/router/0.log" Jul 08 18:42:42.904634 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:42.904609 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:43.382182 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:43.382156 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wscxz_c6a37a64-1b14-4e6a-a15c-91ed60294d7f/istio-proxy/0.log" Jul 08 18:42:43.566807 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:43.566738 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-29tmb_5ba67f36-15a0-4cd4-9dd5-2376c770a2c9/kube-auth-proxy/0.log" Jul 08 18:42:43.598747 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:43.598716 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vfz68_59a21420-9066-4247-939c-aef787050364/istio-proxy/0.log" Jul 08 18:42:43.625478 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:43.625456 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vfz68_59a21420-9066-4247-939c-aef787050364/istio-proxy/1.log" Jul 08 18:42:43.655412 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:43.655388 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-d8l8b_5d010009-a12f-424c-ae94-96042a1fedd2/payload-pre-processing/0.log" Jul 08 18:42:43.734980 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:43.734951 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d459b6969-8sg97_6c74951d-5706-4a46-9163-5b9d3dc67042/router/0.log" Jul 08 18:42:44.542520 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:44.542490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9hvmv_82f9ee10-112d-4d87-91b3-9619e72c1c03/konnectivity-agent/0.log" limit=1024 Jul 08 18:42:46.548547 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:46.548518 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-28nrl_59552106-b49a-4d61-8dd5-d6cea96444d1/migrator/0.log" Jul 08 18:42:46.570690 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:46.570665 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-28nrl_59552106-b49a-4d61-8dd5-d6cea96444d1/graceful-termination/0.log" Jul 08 18:42:46.908019 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:46.907989 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wf66x_7c57f641-7b79-42b9-a796-35eb35a22fc6/kube-storage-version-migrator-operator/1.log" Jul 08 18:42:46.908781 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:46.908764 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wf66x_7c57f641-7b79-42b9-a796-35eb35a22fc6/kube-storage-version-migrator-operator/0.log" Jul 08 18:42:48.008257 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:48.008229 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-46qjq_319f6f85-dcd0-4060-89f5-8c32c347ba0d/kube-multus-additional-cni-plugins/0.log" Jul 08 18:42:48.038099 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:48.038071 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-46qjq_319f6f85-dcd0-4060-89f5-8c32c347ba0d/egress-router-binary-copy/0.log" Jul 08 18:42:48.068047 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:48.068014 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-46qjq_319f6f85-dcd0-4060-89f5-8c32c347ba0d/cni-plugins/0.log" Jul 08 18:42:48.094696 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:48.094672 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-46qjq_319f6f85-dcd0-4060-89f5-8c32c347ba0d/bond-cni-plugin/0.log" Jul 08 18:42:48.127741 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:48.127720 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-46qjq_319f6f85-dcd0-4060-89f5-8c32c347ba0d/routeoverride-cni/0.log" Jul 08 18:42:48.159458 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:48.159435 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-46qjq_319f6f85-dcd0-4060-89f5-8c32c347ba0d/whereabouts-cni-bincopy/0.log" Jul 08 18:42:48.175415 ip-10-0-141-56 kubenswrapper[2580]: I0708 18:42:48.175389 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-46qjq_319f6f85-dcd0-4060-89f5-8c32c347ba0d/whereabouts-cni/0.log"