Jul 09 18:43:29.133563 ip-10-0-134-249 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 18:43:29.133579 ip-10-0-134-249 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 18:43:29.133589 ip-10-0-134-249 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 18:43:29.133915 ip-10-0-134-249 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 18:43:39.290305 ip-10-0-134-249 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 18:43:39.290321 ip-10-0-134-249 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 582d0b1a7d054f04a5c73044ab276437 -- Jul 09 18:45:13.368223 ip-10-0-134-249 systemd[1]: Starting Kubernetes Kubelet... Jul 09 18:45:13.817587 ip-10-0-134-249 kubenswrapper[2587]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:45:13.817587 ip-10-0-134-249 kubenswrapper[2587]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 18:45:13.817587 ip-10-0-134-249 kubenswrapper[2587]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:45:13.817587 ip-10-0-134-249 kubenswrapper[2587]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 18:45:13.817587 ip-10-0-134-249 kubenswrapper[2587]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:45:13.817587 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.816260 2587 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824894 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824906 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824910 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824914 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824917 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824923 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824927 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824929 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824932 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824935 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824938 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824956 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824959 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824964 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824969 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824972 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824975 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824977 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824983 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:45:13.828653 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824986 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824989 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824991 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824996 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.824999 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825002 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825006 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825009 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825012 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825015 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825017 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825023 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825025 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825028 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825031 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825033 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825036 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825039 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825043 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:45:13.832393 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825046 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825049 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825051 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825054 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825060 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825063 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825065 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825068 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825071 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825073 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825076 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825079 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825082 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825085 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825088 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825091 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825096 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825098 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825101 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825104 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:45:13.835004 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825107 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825110 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825112 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825116 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825118 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825122 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825124 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825127 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825130 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825137 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825141 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825144 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825147 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825151 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825154 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825157 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825161 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825164 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825168 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:45:13.838472 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825171 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825176 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825179 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825182 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825184 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825187 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825190 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825192 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825195 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825198 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825200 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825203 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825206 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825208 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825213 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825216 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825219 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825222 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825225 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825227 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:45:13.841360 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825230 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.825233 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827157 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827167 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827171 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827174 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827178 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827181 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827185 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827192 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827195 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827199 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827202 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827206 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827210 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827214 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827218 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827221 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827224 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827227 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:45:13.845040 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827230 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827236 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827239 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827242 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827245 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827247 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827250 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827253 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827256 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827259 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827262 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827265 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827268 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827273 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827276 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827278 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827281 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827284 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827287 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:45:13.847687 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827290 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827295 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827298 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827302 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827305 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827308 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827314 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827316 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827319 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827322 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827325 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827328 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827331 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827334 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827336 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827339 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827342 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827345 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827351 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827353 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:45:13.851034 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827356 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827359 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827362 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827365 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827368 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827370 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827373 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827376 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827378 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827381 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827384 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827391 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827394 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827396 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827399 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827402 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827404 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827407 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827410 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827413 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:45:13.853801 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827450 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827472 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827478 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827484 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827495 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827498 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827519 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827522 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827525 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827680 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827685 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827689 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827691 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827694 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827697 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827699 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827702 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827705 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827707 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:45:13.857582 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827710 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827713 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.827716 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828141 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828148 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828150 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828153 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828155 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828158 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828161 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828164 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828167 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828172 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828176 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828179 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828182 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828185 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828188 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828190 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:45:13.860588 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828194 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828196 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828199 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828203 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828206 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828209 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828211 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828214 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828216 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828219 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828222 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828224 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828226 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828229 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828232 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828234 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828237 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828240 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828242 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828245 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:45:13.864225 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828247 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828250 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828252 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828255 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828258 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828260 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828263 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828266 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828268 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828270 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828274 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828278 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828281 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828284 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828287 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828289 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828292 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828294 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828297 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:45:13.866930 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828299 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828302 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828304 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828307 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828309 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828312 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828315 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828318 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828321 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828325 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828327 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828329 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828332 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828334 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828337 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828339 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828342 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828344 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828347 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828349 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:45:13.870695 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828352 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828354 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828356 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828359 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828361 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828364 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828366 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828369 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828373 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828376 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828379 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828381 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828384 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828387 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828389 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828392 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828394 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828397 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828399 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828401 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:45:13.894855 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828404 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828407 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828409 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.828412 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828476 2587 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828483 2587 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828489 2587 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828493 2587 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828497 2587 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828500 2587 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828504 2587 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828508 2587 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828511 2587 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828516 2587 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828520 2587 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828523 2587 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828526 2587 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828529 2587 flags.go:64] FLAG: --cgroup-root="" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828532 2587 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828535 2587 flags.go:64] FLAG: --client-ca-file="" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828539 2587 flags.go:64] FLAG: --cloud-provider="external" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828542 2587 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828547 2587 flags.go:64] FLAG: --cluster-domain="" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828550 2587 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 18:45:13.912224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828553 2587 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 18:45:13.899281 ip-10-0-134-249 systemd[1]: Started Kubernetes Kubelet. Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828556 2587 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828559 2587 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828563 2587 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828566 2587 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828569 2587 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828573 2587 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828576 2587 flags.go:64] FLAG: --contention-profiling="false" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828579 2587 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828582 2587 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828585 2587 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828588 2587 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828592 2587 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828594 2587 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828597 2587 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828600 2587 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828603 2587 flags.go:64] FLAG: --enable-server="true" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828606 2587 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828610 2587 flags.go:64] FLAG: --event-burst="100" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828613 2587 flags.go:64] FLAG: --event-qps="50" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828615 2587 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828622 2587 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828624 2587 flags.go:64] FLAG: --eviction-hard="" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828628 2587 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 18:45:14.133990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828631 2587 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828634 2587 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828637 2587 flags.go:64] FLAG: --eviction-soft="" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828639 2587 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828642 2587 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828646 2587 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828649 2587 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828652 2587 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828655 2587 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828658 2587 flags.go:64] FLAG: --feature-gates="" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828661 2587 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828664 2587 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828668 2587 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828671 2587 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828674 2587 flags.go:64] FLAG: --healthz-port="10248" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828677 2587 flags.go:64] FLAG: --help="false" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828680 2587 flags.go:64] FLAG: --hostname-override="ip-10-0-134-249.ec2.internal" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828683 2587 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828686 2587 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828689 2587 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828692 2587 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828695 2587 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828698 2587 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 18:45:14.150734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828701 2587 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828705 2587 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828708 2587 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828711 2587 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828714 2587 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828717 2587 flags.go:64] FLAG: --kube-reserved="" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828721 2587 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828724 2587 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828727 2587 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828730 2587 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828733 2587 flags.go:64] FLAG: --lock-file="" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828735 2587 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828738 2587 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828741 2587 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828746 2587 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828750 2587 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828753 2587 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828756 2587 flags.go:64] FLAG: --logging-format="text" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828759 2587 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828762 2587 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828765 2587 flags.go:64] FLAG: --manifest-url="" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828768 2587 flags.go:64] FLAG: --manifest-url-header="" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828772 2587 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828775 2587 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828779 2587 flags.go:64] FLAG: --max-pods="110" Jul 09 18:45:14.151638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828782 2587 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828785 2587 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828788 2587 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828791 2587 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828793 2587 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828796 2587 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828799 2587 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828806 2587 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828809 2587 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828812 2587 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828815 2587 flags.go:64] FLAG: --pod-cidr="" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828818 2587 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828822 2587 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828827 2587 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828830 2587 flags.go:64] FLAG: --pods-per-core="0" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828833 2587 flags.go:64] FLAG: --port="10250" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828836 2587 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828839 2587 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0294133fc07e5151f" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828842 2587 flags.go:64] FLAG: --qos-reserved="" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828845 2587 flags.go:64] FLAG: --read-only-port="10255" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828850 2587 flags.go:64] FLAG: --register-node="true" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828853 2587 flags.go:64] FLAG: --register-with-taints="" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828856 2587 flags.go:64] FLAG: --registry-burst="10" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828860 2587 flags.go:64] FLAG: --registry-qps="5" Jul 09 18:45:14.152824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828863 2587 flags.go:64] FLAG: --reserved-cpus="" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828866 2587 flags.go:64] FLAG: --reserved-memory="" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828869 2587 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828872 2587 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828875 2587 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828878 2587 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828881 2587 flags.go:64] FLAG: --runonce="false" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828884 2587 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828887 2587 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828890 2587 flags.go:64] FLAG: --seccomp-default="false" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828893 2587 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828895 2587 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828898 2587 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828901 2587 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828904 2587 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828907 2587 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828910 2587 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828913 2587 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828916 2587 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828919 2587 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828921 2587 flags.go:64] FLAG: --system-cgroups="" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828926 2587 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828931 2587 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828933 2587 flags.go:64] FLAG: --tls-cert-file="" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828936 2587 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 18:45:14.153634 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828958 2587 flags.go:64] FLAG: --tls-min-version="" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828962 2587 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828964 2587 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828969 2587 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828972 2587 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828975 2587 flags.go:64] FLAG: --v="2" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828979 2587 flags.go:64] FLAG: --version="false" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828990 2587 flags.go:64] FLAG: --vmodule="" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.828998 2587 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.829002 2587 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829105 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829108 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829111 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829114 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829117 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829120 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829125 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829128 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829131 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829134 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829136 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829139 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:45:14.154438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829142 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829144 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829147 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829150 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829152 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829155 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829159 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829161 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829164 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829166 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829169 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829171 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829174 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829178 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829181 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829184 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829186 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829189 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829192 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:45:14.155251 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829195 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829197 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829200 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829202 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829205 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829207 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829210 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829213 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829215 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829217 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829221 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829225 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829228 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829230 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829233 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829235 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829238 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829241 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829244 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829247 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:45:14.156062 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829250 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829252 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829255 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829257 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829260 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829262 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829266 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829269 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829271 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829274 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829276 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829279 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829282 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829286 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829288 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829291 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829294 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829296 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829299 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829301 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:45:14.156861 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829304 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829306 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829309 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829311 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829314 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829316 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829319 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829321 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829324 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829326 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829329 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829332 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829335 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829337 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829340 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829343 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829346 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829350 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829355 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:45:14.157646 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829358 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829360 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829363 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829365 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829368 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829371 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829374 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829376 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.829379 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.829387 2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.836133 2587 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.836154 2587 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.836170 2587 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.836176 2587 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836228 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:45:14.158462 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836233 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836236 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836239 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836242 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836245 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836248 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836250 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836255 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836259 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836262 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836265 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836269 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836272 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836274 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836278 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836281 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836284 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836286 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836289 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:45:14.159267 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836292 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836296 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836299 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836302 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836305 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836308 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836311 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836314 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836316 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836319 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836322 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836324 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836328 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836331 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836335 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836337 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836340 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836342 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836345 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:45:14.160417 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836348 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836350 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836353 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836355 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836358 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836360 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836363 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836365 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836368 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836370 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836373 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836375 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836378 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836380 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836383 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836385 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836388 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836390 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836394 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836397 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836399 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:45:14.161202 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836402 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836404 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836407 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836409 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836412 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836414 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836416 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836419 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836422 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836424 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836427 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836430 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836432 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836435 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836438 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836440 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836443 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836445 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836448 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836450 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:45:14.162059 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836453 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836455 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836458 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836461 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836463 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836466 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836468 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836470 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836475 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836478 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836482 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836484 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836487 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836489 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836492 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836495 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836497 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836500 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:45:14.162878 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836503 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.836508 2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836613 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836617 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836619 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836622 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836626 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836629 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836631 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836634 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836639 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836642 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836644 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836647 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836649 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:45:14.163680 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836652 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836654 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836656 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836659 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836661 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836664 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836667 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836671 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836675 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836678 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836681 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836684 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836687 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836689 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836692 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836694 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836696 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836699 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836702 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:45:14.164471 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836704 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836707 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836709 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836712 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836715 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836717 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836719 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836722 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836726 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836728 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836731 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836734 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836736 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836739 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836741 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836744 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836746 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836748 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836751 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836753 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:45:14.165314 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836756 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836760 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836762 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836765 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836770 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836773 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836776 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836778 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836781 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836783 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836786 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836788 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836790 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836793 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836795 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836798 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836800 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836803 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836805 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836809 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:45:14.166477 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836814 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836817 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836819 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836822 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836824 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836827 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836830 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836832 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836835 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836837 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836840 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836842 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836845 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836849 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836851 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836854 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836856 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836859 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836862 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:45:14.171595 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836864 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836868 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836872 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836874 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836877 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836880 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836883 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:13.836886 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.836890 2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.837020 2587 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.838994 2587 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.841542 2587 server.go:1019] "Starting client certificate rotation" Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.841643 2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:45:14.172392 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.841681 2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.844178 2587 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.849311 2587 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.860112 2587 log.go:25] "Validated CRI v1 runtime API" Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.862452 2587 log.go:25] "Validated CRI v1 image API" Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.863231 2587 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.869178 2587 fs.go:135] Filesystem UUIDs: map[55b5854c-fea0-4633-9eb8-df800ef5bc5b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 96499655-45ef-424c-a473-10da1d50288f:/dev/nvme0n1p4] Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.869191 2587 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 18:45:14.279585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.875117 2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.875112 2587 manager.go:216] Machine: {Timestamp:2026-07-09 18:45:13.873718869 +0000 UTC m=+0.374677919 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097140 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ca164ff66bbe7c15cfb0c450b1ac2 SystemUUID:ec2ca164-ff66-bbe7-c15c-fb0c450b1ac2 BootID:582d0b1a-7d05-4f04-a5c7-3044ab276437 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:7a:6d:92:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:7a:6d:92:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:1a:d9:a0:61:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.875221 2587 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.875326 2587 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.878351 2587 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.878372 2587 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-249.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.879085 2587 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.879096 2587 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.879103 2587 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.879578 2587 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.880808 2587 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.884548 2587 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.884654 2587 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.884681 2587 kubelet.go:486] "Attempting to sync node with API server" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.884690 2587 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.884705 2587 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.884713 2587 kubelet.go:398] "Adding apiserver pod source" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.884726 2587 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 18:45:14.280328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.885094 2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.885114 2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.887309 2587 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889114 2587 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889141 2587 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889284 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889298 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889303 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889309 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889317 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889323 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889881 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889888 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889894 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.889900 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.894799 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.894816 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.895624 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.895642 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.898612 2587 server.go:1268] "Started kubelet" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.899100 2587 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.900485 2587 server.go:310] "Adding debug handlers to kubelet server" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.903006 2587 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.903035 2587 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.912582 2587 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.913240 2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.913425 2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.913453 2587 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-249.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.919938 2587 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.919966 2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.920090 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-249.ec2.internal\" not found" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.920687 2587 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.920687 2587 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.920712 2587 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.920782 2587 reconstruct.go:97] "Volume reconstruction finished" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.920792 2587 reconciler.go:29] "Reconciler: start to sync state" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.924532 2587 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.924604 2587 factory.go:55] Registering systemd factory Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.924616 2587 factory.go:223] Registration of the systemd container factory successfully Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.924857 2587 factory.go:153] Registering CRI-O factory Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.924873 2587 factory.go:223] Registration of the crio container factory successfully Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.924923 2587 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.924965 2587 factory.go:103] Registering Raw factory Jul 09 18:45:14.281100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.924978 2587 manager.go:1196] Started watching for new ooms in manager Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.924599 2587 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-249.ec2.internal.18c0b368833e187e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-249.ec2.internal,UID:ip-10-0-134-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-249.ec2.internal,},FirstTimestamp:2026-07-09 18:45:13.898571902 +0000 UTC m=+0.399530935,LastTimestamp:2026-07-09 18:45:13.898571902 +0000 UTC m=+0.399530935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-249.ec2.internal,}" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.926457 2587 manager.go:349] Starting recovery of all containers Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.930048 2587 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.930160 2587 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.934484 2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zg8mh" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.942015 2587 manager.go:354] Recovery completed Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.942112 2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zg8mh" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.949064 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.948318 2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.952780 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.952810 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.952825 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.953462 2587 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.953472 2587 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.953490 2587 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.953506 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.954669 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.954688 2587 policy_none.go:49] "None policy: Start" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.954702 2587 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.954704 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.954715 2587 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.954728 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.955225 2587 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-249.ec2.internal.18c0b368867977e3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-249.ec2.internal,UID:ip-10-0-134-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-249.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-249.ec2.internal,},FirstTimestamp:2026-07-09 18:45:13.952794595 +0000 UTC m=+0.453753623,LastTimestamp:2026-07-09 18:45:13.952794595 +0000 UTC m=+0.453753623,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-249.ec2.internal,}" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.955874 2587 policy_none.go:47] "Start" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.999486 2587 manager.go:337] "Starting Device Plugin manager" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:13.999523 2587 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.999537 2587 server.go:87] "Starting device plugin registration server" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.999765 2587 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.999775 2587 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.999910 2587 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:13.999994 2587 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.000002 2587 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 18:45:14.282264 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:14.000426 2587 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:14.000475 2587 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-249.ec2.internal\" not found" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.058997 2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.059030 2587 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.059049 2587 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:14.059079 2587 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.063202 2587 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.100589 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.101388 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.101416 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.101429 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.101448 2587 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-249.ec2.internal" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.114802 2587 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-249.ec2.internal" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:14.114818 2587 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-249.ec2.internal\": node \"ip-10-0-134-249.ec2.internal\" not found" Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.159536 2587 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal"] Jul 09 18:45:14.283469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:14.159692 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:45:15.485246 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.484921 2587 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 18:45:15.485246 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.485039 2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 18:40:14 +0000 UTC" deadline="2028-04-03 01:06:56.773194334 +0000 UTC" Jul 09 18:45:15.485246 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.485067 2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15198h21m41.288131514s" Jul 09 18:45:15.485246 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:15.485166 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-249.ec2.internal\" not found" Jul 09 18:45:15.485915 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.485283 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:45:15.485915 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.485312 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:45:15.485915 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.485324 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:45:15.487068 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.487055 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:45:15.487137 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.487122 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.487187 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.487161 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:45:15.488115 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.488097 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:45:15.488205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.488127 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:45:15.488205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.488147 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:45:15.488205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.488194 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:45:15.488306 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.488213 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:45:15.488306 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.488223 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:45:15.489590 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.489573 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.489694 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.489602 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:45:15.490503 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.490487 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:45:15.490572 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.490519 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:45:15.490572 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.490530 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:45:15.501464 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.501444 2587 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:45:15.501584 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.501568 2587 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:45:15.506237 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.506222 2587 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:45:15.521042 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.521024 2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.522865 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.522849 2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.527749 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.527733 2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.537895 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.537879 2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:45:15.540894 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.540879 2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:45:15.541836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.541817 2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:45:15.541885 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:15.541857 2587 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.541885 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.541868 2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.553016 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.552996 2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:45:15.553077 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:15.553036 2587 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-249.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.585990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.585967 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/226d8d2c98aa25e6e0873bbabeb7b26f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal\" (UID: \"226d8d2c98aa25e6e0873bbabeb7b26f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.586045 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.586000 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/226d8d2c98aa25e6e0873bbabeb7b26f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal\" (UID: \"226d8d2c98aa25e6e0873bbabeb7b26f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.586045 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.586030 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfa3c3be93b6e0164ff4342e8421b411-config\") pod \"kube-apiserver-proxy-ip-10-0-134-249.ec2.internal\" (UID: \"bfa3c3be93b6e0164ff4342e8421b411\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.687284 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.687256 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/226d8d2c98aa25e6e0873bbabeb7b26f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal\" (UID: \"226d8d2c98aa25e6e0873bbabeb7b26f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.687284 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.687285 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfa3c3be93b6e0164ff4342e8421b411-config\") pod \"kube-apiserver-proxy-ip-10-0-134-249.ec2.internal\" (UID: \"bfa3c3be93b6e0164ff4342e8421b411\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.687440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.687302 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/226d8d2c98aa25e6e0873bbabeb7b26f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal\" (UID: \"226d8d2c98aa25e6e0873bbabeb7b26f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.687440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.687339 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/226d8d2c98aa25e6e0873bbabeb7b26f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal\" (UID: \"226d8d2c98aa25e6e0873bbabeb7b26f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.687440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.687380 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/226d8d2c98aa25e6e0873bbabeb7b26f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal\" (UID: \"226d8d2c98aa25e6e0873bbabeb7b26f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.687440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.687385 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfa3c3be93b6e0164ff4342e8421b411-config\") pod \"kube-apiserver-proxy-ip-10-0-134-249.ec2.internal\" (UID: \"bfa3c3be93b6e0164ff4342e8421b411\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.843545 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.843462 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" Jul 09 18:45:15.843545 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:15.843476 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" Jul 09 18:45:16.485218 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.485191 2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 18:45:16.485693 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.485664 2587 apiserver.go:52] "Watching apiserver" Jul 09 18:45:16.491941 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.491908 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa3c3be93b6e0164ff4342e8421b411.slice/crio-2fac0cc9c340cbbee1a95d4b58d5ca60565e1a8d81c47cbffd03da428bd8a2f6 WatchSource:0}: Error finding container 2fac0cc9c340cbbee1a95d4b58d5ca60565e1a8d81c47cbffd03da428bd8a2f6: Status 404 returned error can't find the container with id 2fac0cc9c340cbbee1a95d4b58d5ca60565e1a8d81c47cbffd03da428bd8a2f6 Jul 09 18:45:16.492524 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.492504 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226d8d2c98aa25e6e0873bbabeb7b26f.slice/crio-f0c4da797b81fa712f024e3a1f278fa11519b4b91c16e9deca2b904674259741 WatchSource:0}: Error finding container f0c4da797b81fa712f024e3a1f278fa11519b4b91c16e9deca2b904674259741: Status 404 returned error can't find the container with id f0c4da797b81fa712f024e3a1f278fa11519b4b91c16e9deca2b904674259741 Jul 09 18:45:16.493872 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.493854 2587 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 18:45:16.495179 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.495155 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-qpr66","kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g","openshift-cluster-node-tuning-operator/tuned-hjd48","openshift-dns/node-resolver-r2x85","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal","openshift-multus/multus-additional-cni-plugins-j4rwj","openshift-multus/multus-fvbjs","openshift-multus/network-metrics-daemon-b62dq","kube-system/global-pull-secret-syncer-vf5zs","kube-system/konnectivity-agent-bg9n2","openshift-image-registry/node-ca-dh7hx","openshift-network-diagnostics/network-check-target-l5vkd","openshift-network-operator/iptables-alerter-lfllq"] Jul 09 18:45:16.497002 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.496949 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:45:16.497494 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.497469 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.498508 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.498486 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:16.498602 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.498577 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:16.499248 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.499226 2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:45:16.499833 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.499812 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.500343 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.500325 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 18:45:16.500470 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.500407 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 18:45:16.500470 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.500413 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 18:45:16.500677 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.500657 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 18:45:16.501551 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.501530 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 18:45:16.501729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.501712 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.502664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.502641 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vcr7d\"" Jul 09 18:45:16.502767 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.502738 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 18:45:16.503099 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.503082 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.503576 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.503557 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 18:45:16.503660 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.503648 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 18:45:16.504248 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.504227 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q67rm\"" Jul 09 18:45:16.504333 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.504319 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.504433 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.504414 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jqwrx\"" Jul 09 18:45:16.504772 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.504752 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:16.504851 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.504784 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:16.505658 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.505640 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.506108 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.506088 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 18:45:16.506191 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.506127 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bpqpt\"" Jul 09 18:45:16.507031 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.507004 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.510678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.510184 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 18:45:16.510678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.510285 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:16.510678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.510414 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t746c\"" Jul 09 18:45:16.510678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.510465 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 18:45:16.510678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.510498 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:16.510678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.510531 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-npdwd\"" Jul 09 18:45:16.510678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.510659 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5dkss\"" Jul 09 18:45:16.511155 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.511131 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.511334 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.511315 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 18:45:16.511522 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.511507 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 18:45:16.511639 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.511596 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 18:45:16.512379 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.512359 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 18:45:16.512609 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.512584 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 18:45:16.512713 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.512687 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 18:45:16.512793 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.512722 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.512854 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.512783 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:16.512854 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.512591 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 18:45:16.514255 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.514191 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.514255 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.514240 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 18:45:16.514421 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.514287 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qchhs\"" Jul 09 18:45:16.514421 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.514243 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 18:45:16.516734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.516713 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:16.516815 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.516779 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:16.516856 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.516823 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 18:45:16.517097 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.517077 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 18:45:16.517195 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.517083 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 18:45:16.517195 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.517171 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dmh2t\"" Jul 09 18:45:16.521782 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.521743 2587 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 18:45:16.549040 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.549008 2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l2qbr" Jul 09 18:45:16.558524 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.558509 2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l2qbr" Jul 09 18:45:16.604694 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604666 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-system-cni-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604702 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc77411f-12b3-40e2-b71a-b338eabd5065-konnectivity-ca\") pod \"konnectivity-agent-bg9n2\" (UID: \"dc77411f-12b3-40e2-b71a-b338eabd5065\") " pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604723 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604746 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-slash\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604769 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-run\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604791 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-kubelet\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604806 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604822 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-conf-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.604836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604837 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc77411f-12b3-40e2-b71a-b338eabd5065-agent-certs\") pod \"konnectivity-agent-bg9n2\" (UID: \"dc77411f-12b3-40e2-b71a-b338eabd5065\") " pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604856 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604909 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604929 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-var-lib-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604961 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604982 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-node-log\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.604996 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-cni-netd\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605017 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-env-overrides\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605047 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fb5d87d-2207-4215-900e-9f5cbde935de-hosts-file\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605068 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-hostroot\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605087 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfg8w\" (UniqueName: \"kubernetes.io/projected/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-kube-api-access-zfg8w\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605127 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysctl-conf\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605167 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-etc-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605200 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm8nc\" (UniqueName: \"kubernetes.io/projected/7fb5d87d-2207-4215-900e-9f5cbde935de-kube-api-access-nm8nc\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605226 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/542ce291-5b21-4361-b4a1-58e67e29c530-host\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605253 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-cnibin\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605287 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-var-lib-kubelet\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605312 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fb5d87d-2207-4215-900e-9f5cbde935de-tmp-dir\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605336 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-system-cni-dir\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605388 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605416 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605440 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-sys-fs\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605463 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-host\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605496 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605522 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-os-release\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605540 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-log-socket\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605555 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-systemd-units\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605571 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysctl-d\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605586 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-sys\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.605799 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605617 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-socket-dir-parent\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605650 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-lib-modules\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605690 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605717 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovnkube-config\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605747 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-iptables-alerter-script\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605773 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88545bfb-fc18-4f99-ab53-894af89974a2-dbus\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605809 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-socket-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605827 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovnkube-script-lib\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605843 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrj5h\" (UniqueName: \"kubernetes.io/projected/f1b9b361-c850-40cd-af98-cd00992eb6d3-kube-api-access-vrj5h\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605858 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hrxt\" (UniqueName: \"kubernetes.io/projected/542ce291-5b21-4361-b4a1-58e67e29c530-kube-api-access-2hrxt\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605874 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-etc-selinux\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605890 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-tuned\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605904 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e3a413-5693-40b5-9241-6bf14e6b3b3f-tmp\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605930 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-cni-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605967 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-netns\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.605989 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-cni-multus\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.606451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606018 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-kubelet\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606040 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/542ce291-5b21-4361-b4a1-58e67e29c530-serviceca\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606056 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-kubernetes\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606070 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606084 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-systemd\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606111 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-cni-bin\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606134 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npbmw\" (UniqueName: \"kubernetes.io/projected/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-kube-api-access-npbmw\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606151 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-os-release\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606179 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-multus-certs\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606202 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh9vt\" (UniqueName: \"kubernetes.io/projected/adfe3fbd-54e0-4772-8240-4afdd653ac76-kube-api-access-lh9vt\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606227 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzb7j\" (UniqueName: \"kubernetes.io/projected/f30cc372-130d-417c-a1c4-2754e9d9f80d-kube-api-access-vzb7j\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606266 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-daemon-config\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606289 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-modprobe-d\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606321 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysconfig\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606345 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-device-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606383 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45mmq\" (UniqueName: \"kubernetes.io/projected/92cf6223-3b85-4530-b4cd-7b2cf230a717-kube-api-access-45mmq\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.607105 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606418 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606448 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-k8s-cni-cncf-io\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606466 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-cni-bin\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606483 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88545bfb-fc18-4f99-ab53-894af89974a2-kubelet-config\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606505 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovn-node-metrics-cert\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606535 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-host-slash\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606563 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-cni-binary-copy\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606599 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-etc-kubernetes\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606630 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-cnibin\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606666 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-registration-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606691 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-systemd\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606725 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-ovn\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606757 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dswl\" (UniqueName: \"kubernetes.io/projected/97e3a413-5693-40b5-9241-6bf14e6b3b3f-kube-api-access-2dswl\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.607598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.606782 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-run-netns\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.707635 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707613 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-cni-binary-copy\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707640 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-etc-kubernetes\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707656 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-cnibin\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707676 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-registration-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707700 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-systemd\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707722 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-ovn\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707748 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dswl\" (UniqueName: \"kubernetes.io/projected/97e3a413-5693-40b5-9241-6bf14e6b3b3f-kube-api-access-2dswl\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707752 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-etc-kubernetes\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707771 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-run-netns\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.707788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707775 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-registration-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707808 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-system-cni-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707813 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-ovn\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707827 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-run-netns\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707869 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-system-cni-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707877 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-systemd\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707911 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc77411f-12b3-40e2-b71a-b338eabd5065-konnectivity-ca\") pod \"konnectivity-agent-bg9n2\" (UID: \"dc77411f-12b3-40e2-b71a-b338eabd5065\") " pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707924 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-cnibin\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707943 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.707988 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-slash\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708014 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-run\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708037 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-kubelet\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708060 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708066 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-slash\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708085 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-conf-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708109 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc77411f-12b3-40e2-b71a-b338eabd5065-agent-certs\") pod \"konnectivity-agent-bg9n2\" (UID: \"dc77411f-12b3-40e2-b71a-b338eabd5065\") " pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708111 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-kubelet\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708130 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.708300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708132 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708170 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708192 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-var-lib-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708204 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-conf-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708213 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708244 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-node-log\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708247 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708248 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-run\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708288 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-node-log\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708317 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-cni-netd\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.708336 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708342 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-env-overrides\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708347 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-cni-binary-copy\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708366 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fb5d87d-2207-4215-900e-9f5cbde935de-hosts-file\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.708396 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs podName:f30cc372-130d-417c-a1c4-2754e9d9f80d nodeName:}" failed. No retries permitted until 2026-07-09 18:45:17.208375386 +0000 UTC m=+3.709334424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs") pod "network-metrics-daemon-b62dq" (UID: "f30cc372-130d-417c-a1c4-2754e9d9f80d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708413 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fb5d87d-2207-4215-900e-9f5cbde935de-hosts-file\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708415 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-hostroot\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.709142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708460 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfg8w\" (UniqueName: \"kubernetes.io/projected/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-kube-api-access-zfg8w\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708493 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysctl-conf\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708519 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-etc-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708526 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc77411f-12b3-40e2-b71a-b338eabd5065-konnectivity-ca\") pod \"konnectivity-agent-bg9n2\" (UID: \"dc77411f-12b3-40e2-b71a-b338eabd5065\") " pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708555 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm8nc\" (UniqueName: \"kubernetes.io/projected/7fb5d87d-2207-4215-900e-9f5cbde935de-kube-api-access-nm8nc\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708580 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/542ce291-5b21-4361-b4a1-58e67e29c530-host\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708588 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-hostroot\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708612 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-cnibin\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708636 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-var-lib-kubelet\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708641 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-var-lib-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708671 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fb5d87d-2207-4215-900e-9f5cbde935de-tmp-dir\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708695 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-var-lib-kubelet\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708700 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-system-cni-dir\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708707 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysctl-conf\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708738 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-etc-openvswitch\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708741 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708751 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-system-cni-dir\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.709818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708746 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-cni-netd\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708761 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708764 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708807 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708833 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-sys-fs\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708859 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-host\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708885 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708893 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-env-overrides\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708941 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-cnibin\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.708954 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-sys-fs\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709035 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-host\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.709055 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709056 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fb5d87d-2207-4215-900e-9f5cbde935de-tmp-dir\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709091 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-os-release\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709105 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/542ce291-5b21-4361-b4a1-58e67e29c530-host\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709095 2587 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709161 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-log-socket\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.710691 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.709171 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret podName:88545bfb-fc18-4f99-ab53-894af89974a2 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:17.20915421 +0000 UTC m=+3.710113242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret") pod "global-pull-secret-syncer-vf5zs" (UID: "88545bfb-fc18-4f99-ab53-894af89974a2") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709190 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adfe3fbd-54e0-4772-8240-4afdd653ac76-os-release\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709227 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-log-socket\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709267 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-systemd-units\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709326 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysctl-d\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709346 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adfe3fbd-54e0-4772-8240-4afdd653ac76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709352 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-systemd-units\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709388 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-sys\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709451 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-socket-dir-parent\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709476 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysctl-d\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709489 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-sys\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709498 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-lib-modules\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709512 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-socket-dir-parent\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709554 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709577 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovnkube-config\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709581 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-lib-modules\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709592 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-iptables-alerter-script\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.711714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709608 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88545bfb-fc18-4f99-ab53-894af89974a2-dbus\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709616 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709631 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-socket-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709657 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovnkube-script-lib\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709699 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrj5h\" (UniqueName: \"kubernetes.io/projected/f1b9b361-c850-40cd-af98-cd00992eb6d3-kube-api-access-vrj5h\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709721 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88545bfb-fc18-4f99-ab53-894af89974a2-dbus\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709734 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hrxt\" (UniqueName: \"kubernetes.io/projected/542ce291-5b21-4361-b4a1-58e67e29c530-kube-api-access-2hrxt\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709766 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-etc-selinux\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709792 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-tuned\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709847 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e3a413-5693-40b5-9241-6bf14e6b3b3f-tmp\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709873 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-cni-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709898 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-netns\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709934 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-cni-multus\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.709982 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-kubelet\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710008 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/542ce291-5b21-4361-b4a1-58e67e29c530-serviceca\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710024 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-socket-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710033 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-kubernetes\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710066 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-cni-dir\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710106 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-kubernetes\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710164 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovnkube-config\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710157 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-netns\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710197 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-iptables-alerter-script\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710229 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-cni-multus\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710260 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-kubelet\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710272 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-etc-selinux\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710342 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovnkube-script-lib\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710379 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710426 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-systemd\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710451 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-cni-bin\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710475 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npbmw\" (UniqueName: \"kubernetes.io/projected/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-kube-api-access-npbmw\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710484 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-run-systemd\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710498 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-os-release\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710500 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710549 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-cni-bin\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710551 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-multus-certs\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.712929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710584 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh9vt\" (UniqueName: \"kubernetes.io/projected/adfe3fbd-54e0-4772-8240-4afdd653ac76-kube-api-access-lh9vt\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710588 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-multus-certs\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710554 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-os-release\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710643 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/542ce291-5b21-4361-b4a1-58e67e29c530-serviceca\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710651 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzb7j\" (UniqueName: \"kubernetes.io/projected/f30cc372-130d-417c-a1c4-2754e9d9f80d-kube-api-access-vzb7j\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710678 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-daemon-config\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710720 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-modprobe-d\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710744 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysconfig\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710769 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-device-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710850 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45mmq\" (UniqueName: \"kubernetes.io/projected/92cf6223-3b85-4530-b4cd-7b2cf230a717-kube-api-access-45mmq\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710861 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-sysconfig\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710886 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-modprobe-d\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710906 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710941 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92cf6223-3b85-4530-b4cd-7b2cf230a717-device-dir\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710982 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-k8s-cni-cncf-io\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711013 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-cni-bin\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.710991 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1b9b361-c850-40cd-af98-cd00992eb6d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.713607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711062 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88545bfb-fc18-4f99-ab53-894af89974a2-kubelet-config\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711068 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-run-k8s-cni-cncf-io\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711116 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-host-var-lib-cni-bin\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711159 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88545bfb-fc18-4f99-ab53-894af89974a2-kubelet-config\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711090 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovn-node-metrics-cert\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711225 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-host-slash\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711305 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-host-slash\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.711897 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-multus-daemon-config\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.712839 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e3a413-5693-40b5-9241-6bf14e6b3b3f-tmp\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.712863 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97e3a413-5693-40b5-9241-6bf14e6b3b3f-etc-tuned\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.713110 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc77411f-12b3-40e2-b71a-b338eabd5065-agent-certs\") pod \"konnectivity-agent-bg9n2\" (UID: \"dc77411f-12b3-40e2-b71a-b338eabd5065\") " pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.714125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.713495 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1b9b361-c850-40cd-af98-cd00992eb6d3-ovn-node-metrics-cert\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.717641 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.717619 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:45:16.717756 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.717647 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:45:16.717756 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.717661 2587 projected.go:196] Error preparing data for projected volume kube-api-access-mcfpz for pod openshift-network-diagnostics/network-check-target-l5vkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:16.717756 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:16.717702 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz podName:fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:17.21769148 +0000 UTC m=+3.718650497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mcfpz" (UniqueName: "kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz") pod "network-check-target-l5vkd" (UID: "fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:16.719734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.719647 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dswl\" (UniqueName: \"kubernetes.io/projected/97e3a413-5693-40b5-9241-6bf14e6b3b3f-kube-api-access-2dswl\") pod \"tuned-hjd48\" (UID: \"97e3a413-5693-40b5-9241-6bf14e6b3b3f\") " pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.719860 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.719804 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm8nc\" (UniqueName: \"kubernetes.io/projected/7fb5d87d-2207-4215-900e-9f5cbde935de-kube-api-access-nm8nc\") pod \"node-resolver-r2x85\" (UID: \"7fb5d87d-2207-4215-900e-9f5cbde935de\") " pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.720094 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.720073 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hrxt\" (UniqueName: \"kubernetes.io/projected/542ce291-5b21-4361-b4a1-58e67e29c530-kube-api-access-2hrxt\") pod \"node-ca-dh7hx\" (UID: \"542ce291-5b21-4361-b4a1-58e67e29c530\") " pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.720648 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.720625 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfg8w\" (UniqueName: \"kubernetes.io/projected/651837c0-e0d3-4abe-94aa-d62e0d8cde1e-kube-api-access-zfg8w\") pod \"multus-fvbjs\" (UID: \"651837c0-e0d3-4abe-94aa-d62e0d8cde1e\") " pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.720990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.720961 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzb7j\" (UniqueName: \"kubernetes.io/projected/f30cc372-130d-417c-a1c4-2754e9d9f80d-kube-api-access-vzb7j\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:16.721100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.721077 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrj5h\" (UniqueName: \"kubernetes.io/projected/f1b9b361-c850-40cd-af98-cd00992eb6d3-kube-api-access-vrj5h\") pod \"ovnkube-node-qpr66\" (UID: \"f1b9b361-c850-40cd-af98-cd00992eb6d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.721207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.721190 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh9vt\" (UniqueName: \"kubernetes.io/projected/adfe3fbd-54e0-4772-8240-4afdd653ac76-kube-api-access-lh9vt\") pod \"multus-additional-cni-plugins-j4rwj\" (UID: \"adfe3fbd-54e0-4772-8240-4afdd653ac76\") " pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.721964 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.721942 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npbmw\" (UniqueName: \"kubernetes.io/projected/b7eaf8ff-494a-4a80-a065-77b2b7a1c78f-kube-api-access-npbmw\") pod \"iptables-alerter-lfllq\" (UID: \"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f\") " pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.722525 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.722504 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45mmq\" (UniqueName: \"kubernetes.io/projected/92cf6223-3b85-4530-b4cd-7b2cf230a717-kube-api-access-45mmq\") pod \"aws-ebs-csi-driver-node-xgd2g\" (UID: \"92cf6223-3b85-4530-b4cd-7b2cf230a717\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.809131 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.809107 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" Jul 09 18:45:16.818046 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.818007 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadfe3fbd_54e0_4772_8240_4afdd653ac76.slice/crio-197f66069c301550f960268974fa6d036803b05ebcd8b77bf298e9611002fe44 WatchSource:0}: Error finding container 197f66069c301550f960268974fa6d036803b05ebcd8b77bf298e9611002fe44: Status 404 returned error can't find the container with id 197f66069c301550f960268974fa6d036803b05ebcd8b77bf298e9611002fe44 Jul 09 18:45:16.818924 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.818896 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hjd48" Jul 09 18:45:16.825632 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.825611 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" Jul 09 18:45:16.831087 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.831070 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r2x85" Jul 09 18:45:16.834867 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.834844 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92cf6223_3b85_4530_b4cd_7b2cf230a717.slice/crio-d35360a67450bb49a1e6ec7b6ba71d71027c0bbbe6fda4768fc20919e5578ac6 WatchSource:0}: Error finding container d35360a67450bb49a1e6ec7b6ba71d71027c0bbbe6fda4768fc20919e5578ac6: Status 404 returned error can't find the container with id d35360a67450bb49a1e6ec7b6ba71d71027c0bbbe6fda4768fc20919e5578ac6 Jul 09 18:45:16.836681 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.836665 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lfllq" Jul 09 18:45:16.842478 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.842465 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvbjs" Jul 09 18:45:16.844689 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.844667 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7eaf8ff_494a_4a80_a065_77b2b7a1c78f.slice/crio-b4c59bcaae8fa162b226f2391571224626b73de4352fe9aafa1a8d0edbbe80dd WatchSource:0}: Error finding container b4c59bcaae8fa162b226f2391571224626b73de4352fe9aafa1a8d0edbbe80dd: Status 404 returned error can't find the container with id b4c59bcaae8fa162b226f2391571224626b73de4352fe9aafa1a8d0edbbe80dd Jul 09 18:45:16.847776 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.847759 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:16.849694 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.849552 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651837c0_e0d3_4abe_94aa_d62e0d8cde1e.slice/crio-cecab3c33818cc55c4dcaf7e3730cbaf9e0f03219bdde3b2c85dd69e8d3c8468 WatchSource:0}: Error finding container cecab3c33818cc55c4dcaf7e3730cbaf9e0f03219bdde3b2c85dd69e8d3c8468: Status 404 returned error can't find the container with id cecab3c33818cc55c4dcaf7e3730cbaf9e0f03219bdde3b2c85dd69e8d3c8468 Jul 09 18:45:16.853751 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.853679 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:16.855304 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.855284 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b9b361_c850_40cd_af98_cd00992eb6d3.slice/crio-3cc948c403580b93ab7c73831ab2d7ded466e2ccf53c0d36189a5808aae27528 WatchSource:0}: Error finding container 3cc948c403580b93ab7c73831ab2d7ded466e2ccf53c0d36189a5808aae27528: Status 404 returned error can't find the container with id 3cc948c403580b93ab7c73831ab2d7ded466e2ccf53c0d36189a5808aae27528 Jul 09 18:45:16.857005 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:16.856985 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dh7hx" Jul 09 18:45:16.862521 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:16.862499 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc77411f_12b3_40e2_b71a_b338eabd5065.slice/crio-4e00a23979fd3be449eec19d5e02cdf566f07c10d506776be26ef37ac5d782e3 WatchSource:0}: Error finding container 4e00a23979fd3be449eec19d5e02cdf566f07c10d506776be26ef37ac5d782e3: Status 404 returned error can't find the container with id 4e00a23979fd3be449eec19d5e02cdf566f07c10d506776be26ef37ac5d782e3 Jul 09 18:45:17.214274 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.214232 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:17.214450 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.214299 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:17.214450 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.214440 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:17.214537 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.214506 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret podName:88545bfb-fc18-4f99-ab53-894af89974a2 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:18.214486821 +0000 UTC m=+4.715445858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret") pod "global-pull-secret-syncer-vf5zs" (UID: "88545bfb-fc18-4f99-ab53-894af89974a2") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:17.214607 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.214440 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:17.214660 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.214620 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs podName:f30cc372-130d-417c-a1c4-2754e9d9f80d nodeName:}" failed. No retries permitted until 2026-07-09 18:45:18.214594381 +0000 UTC m=+4.715553421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs") pod "network-metrics-daemon-b62dq" (UID: "f30cc372-130d-417c-a1c4-2754e9d9f80d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:17.314724 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.314686 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:17.315411 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.314917 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:45:17.315411 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.314950 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:45:17.315411 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.314965 2587 projected.go:196] Error preparing data for projected volume kube-api-access-mcfpz for pod openshift-network-diagnostics/network-check-target-l5vkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:17.315411 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:17.315049 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz podName:fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:18.315012737 +0000 UTC m=+4.815971759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcfpz" (UniqueName: "kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz") pod "network-check-target-l5vkd" (UID: "fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:17.495231 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.495019 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dh7hx" event={"ID":"542ce291-5b21-4361-b4a1-58e67e29c530","Type":"ContainerStarted","Data":"78ea3a5debb5c13bddeac8a667eef5a65a5444a3c4962b1f3f60fd5f85a501c5"} Jul 09 18:45:17.505306 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.505271 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"3cc948c403580b93ab7c73831ab2d7ded466e2ccf53c0d36189a5808aae27528"} Jul 09 18:45:17.508084 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.508056 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvbjs" event={"ID":"651837c0-e0d3-4abe-94aa-d62e0d8cde1e","Type":"ContainerStarted","Data":"cecab3c33818cc55c4dcaf7e3730cbaf9e0f03219bdde3b2c85dd69e8d3c8468"} Jul 09 18:45:17.512146 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.512071 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" event={"ID":"92cf6223-3b85-4530-b4cd-7b2cf230a717","Type":"ContainerStarted","Data":"d35360a67450bb49a1e6ec7b6ba71d71027c0bbbe6fda4768fc20919e5578ac6"} Jul 09 18:45:17.515662 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.515587 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hjd48" event={"ID":"97e3a413-5693-40b5-9241-6bf14e6b3b3f","Type":"ContainerStarted","Data":"3b0429b640d9bc6140d4dddd03c513a9319f317e49cfd01c7c96ed19503800e1"} Jul 09 18:45:17.525966 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.525822 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" event={"ID":"226d8d2c98aa25e6e0873bbabeb7b26f","Type":"ContainerStarted","Data":"f0c4da797b81fa712f024e3a1f278fa11519b4b91c16e9deca2b904674259741"} Jul 09 18:45:17.547944 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.547907 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bg9n2" event={"ID":"dc77411f-12b3-40e2-b71a-b338eabd5065","Type":"ContainerStarted","Data":"4e00a23979fd3be449eec19d5e02cdf566f07c10d506776be26ef37ac5d782e3"} Jul 09 18:45:17.560786 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.560581 2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:40:16 +0000 UTC" deadline="2028-02-08 09:57:33.711651634 +0000 UTC" Jul 09 18:45:17.560786 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.560646 2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13887h12m16.151010426s" Jul 09 18:45:17.568665 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.568632 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lfllq" event={"ID":"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f","Type":"ContainerStarted","Data":"b4c59bcaae8fa162b226f2391571224626b73de4352fe9aafa1a8d0edbbe80dd"} Jul 09 18:45:17.575067 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.574963 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r2x85" event={"ID":"7fb5d87d-2207-4215-900e-9f5cbde935de","Type":"ContainerStarted","Data":"b1bcff37af8a1e5e13aeb0162385d52cd9b8fd15106b02d9c2616b6cf32c723c"} Jul 09 18:45:17.581148 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.581072 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerStarted","Data":"197f66069c301550f960268974fa6d036803b05ebcd8b77bf298e9611002fe44"} Jul 09 18:45:17.589164 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:17.589090 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" event={"ID":"bfa3c3be93b6e0164ff4342e8421b411","Type":"ContainerStarted","Data":"2fac0cc9c340cbbee1a95d4b58d5ca60565e1a8d81c47cbffd03da428bd8a2f6"} Jul 09 18:45:18.061175 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.061142 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:18.061385 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.061282 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:18.061957 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.061930 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:18.062072 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.062056 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:18.062218 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.062201 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:18.062337 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.062315 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:18.226838 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.226801 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:18.227061 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.226885 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:18.227061 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.227031 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:18.227189 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.227100 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret podName:88545bfb-fc18-4f99-ab53-894af89974a2 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:20.227080069 +0000 UTC m=+6.728039094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret") pod "global-pull-secret-syncer-vf5zs" (UID: "88545bfb-fc18-4f99-ab53-894af89974a2") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:18.227535 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.227514 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:18.227618 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.227572 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs podName:f30cc372-130d-417c-a1c4-2754e9d9f80d nodeName:}" failed. No retries permitted until 2026-07-09 18:45:20.227557055 +0000 UTC m=+6.728516078 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs") pod "network-metrics-daemon-b62dq" (UID: "f30cc372-130d-417c-a1c4-2754e9d9f80d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:18.327693 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.327594 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:18.327853 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.327810 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:45:18.327853 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.327833 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:45:18.327853 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.327846 2587 projected.go:196] Error preparing data for projected volume kube-api-access-mcfpz for pod openshift-network-diagnostics/network-check-target-l5vkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:18.328040 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:18.327911 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz podName:fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:20.327891004 +0000 UTC m=+6.828850037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcfpz" (UniqueName: "kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz") pod "network-check-target-l5vkd" (UID: "fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:18.561354 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.561282 2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:40:16 +0000 UTC" deadline="2028-02-07 01:09:35.766190448 +0000 UTC" Jul 09 18:45:18.561354 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:18.561322 2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13854h24m17.204873102s" Jul 09 18:45:20.060159 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:20.060126 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:20.060647 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.060285 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:20.060726 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:20.060699 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:20.060827 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.060797 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:20.062953 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:20.062821 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:20.062953 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.062914 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:20.243902 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:20.243638 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:20.243902 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:20.243712 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:20.243902 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.243815 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:20.243902 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.243848 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:20.243902 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.243891 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs podName:f30cc372-130d-417c-a1c4-2754e9d9f80d nodeName:}" failed. No retries permitted until 2026-07-09 18:45:24.24386936 +0000 UTC m=+10.744828395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs") pod "network-metrics-daemon-b62dq" (UID: "f30cc372-130d-417c-a1c4-2754e9d9f80d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:20.243902 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.243915 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret podName:88545bfb-fc18-4f99-ab53-894af89974a2 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:24.243903191 +0000 UTC m=+10.744862213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret") pod "global-pull-secret-syncer-vf5zs" (UID: "88545bfb-fc18-4f99-ab53-894af89974a2") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:20.345004 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:20.344898 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:20.345203 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.345180 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:45:20.345294 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.345208 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:45:20.345294 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.345221 2587 projected.go:196] Error preparing data for projected volume kube-api-access-mcfpz for pod openshift-network-diagnostics/network-check-target-l5vkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:20.345294 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:20.345288 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz podName:fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:24.345267864 +0000 UTC m=+10.846226901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcfpz" (UniqueName: "kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz") pod "network-check-target-l5vkd" (UID: "fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:22.060709 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:22.060676 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:22.061163 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:22.060689 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:22.061163 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:22.060828 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:22.061163 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:22.060811 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:22.061163 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:22.060965 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:22.061163 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:22.061049 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:24.060545 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:24.060510 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:24.061007 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:24.060558 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:24.061007 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.060633 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:24.061007 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.060737 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:24.061007 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:24.060779 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:24.061007 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.060841 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:24.275904 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:24.275870 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:24.276112 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:24.275958 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:24.276112 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.276062 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:24.276112 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.276072 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:24.276282 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.276122 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret podName:88545bfb-fc18-4f99-ab53-894af89974a2 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:32.2761041 +0000 UTC m=+18.777063118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret") pod "global-pull-secret-syncer-vf5zs" (UID: "88545bfb-fc18-4f99-ab53-894af89974a2") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:24.276282 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.276140 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs podName:f30cc372-130d-417c-a1c4-2754e9d9f80d nodeName:}" failed. No retries permitted until 2026-07-09 18:45:32.276129845 +0000 UTC m=+18.777088862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs") pod "network-metrics-daemon-b62dq" (UID: "f30cc372-130d-417c-a1c4-2754e9d9f80d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:24.376474 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:24.376346 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:24.376633 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.376538 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:45:24.376633 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.376557 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:45:24.376633 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.376568 2587 projected.go:196] Error preparing data for projected volume kube-api-access-mcfpz for pod openshift-network-diagnostics/network-check-target-l5vkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:24.376633 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:24.376629 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz podName:fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:32.376610788 +0000 UTC m=+18.877569811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcfpz" (UniqueName: "kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz") pod "network-check-target-l5vkd" (UID: "fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:26.060681 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:26.060604 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:26.061124 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:26.060609 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:26.061124 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:26.060728 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:26.061124 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:26.060740 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:26.061124 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:26.060822 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:26.061124 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:26.060913 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:28.059657 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:28.059622 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:28.060072 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:28.059624 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:28.060072 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:28.059762 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:28.060072 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:28.059622 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:28.060072 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:28.059839 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:28.060072 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:28.059907 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:30.059351 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:30.059317 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:30.059811 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:30.059318 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:30.059811 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:30.059441 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:30.059811 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:30.059496 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:30.059811 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:30.059331 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:30.059811 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:30.059583 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:32.060277 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:32.060085 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:32.060715 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:32.060095 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:32.060715 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.060369 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:32.060715 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.060433 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:32.060715 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:32.060150 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:32.060715 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.060517 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:32.332495 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:32.332423 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:32.332495 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:32.332490 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:32.332702 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.332619 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:32.332702 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.332623 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:32.332702 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.332681 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret podName:88545bfb-fc18-4f99-ab53-894af89974a2 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:48.332663457 +0000 UTC m=+34.833622476 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret") pod "global-pull-secret-syncer-vf5zs" (UID: "88545bfb-fc18-4f99-ab53-894af89974a2") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:32.332702 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.332695 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs podName:f30cc372-130d-417c-a1c4-2754e9d9f80d nodeName:}" failed. No retries permitted until 2026-07-09 18:45:48.332688708 +0000 UTC m=+34.833647724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs") pod "network-metrics-daemon-b62dq" (UID: "f30cc372-130d-417c-a1c4-2754e9d9f80d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:32.433251 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:32.433220 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:32.433418 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.433395 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:45:32.433418 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.433417 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:45:32.433544 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.433429 2587 projected.go:196] Error preparing data for projected volume kube-api-access-mcfpz for pod openshift-network-diagnostics/network-check-target-l5vkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:32.433544 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:32.433495 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz podName:fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:48.433476661 +0000 UTC m=+34.934435677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcfpz" (UniqueName: "kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz") pod "network-check-target-l5vkd" (UID: "fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:34.060721 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:34.060552 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:34.060721 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:34.060667 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:34.061398 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:34.060804 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:34.061398 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:34.060678 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:34.061398 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:34.060872 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:34.061398 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:34.060954 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:36.059813 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:36.059775 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:36.060304 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:36.059782 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:36.060304 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:36.059924 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:36.060304 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:36.060000 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:36.060304 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:36.059793 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:36.060304 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:36.060106 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:37.638881 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:37.638839 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"432b2219640669e55c93d78e809fd915e99d51e3f815a88f73dbf2b99753ceed"} Jul 09 18:45:37.640751 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:37.640656 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvbjs" event={"ID":"651837c0-e0d3-4abe-94aa-d62e0d8cde1e","Type":"ContainerStarted","Data":"baf9ad3b203b76e14b00a96b8b83f1839161cfcb98346a12d6c64ea13f60f246"} Jul 09 18:45:37.642338 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:37.642310 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hjd48" event={"ID":"97e3a413-5693-40b5-9241-6bf14e6b3b3f","Type":"ContainerStarted","Data":"3ac539bde73275c79d42a9c70551ea7564641ad8834e03f8d73d8eef48e2d89f"} Jul 09 18:45:37.643820 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:37.643798 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" event={"ID":"bfa3c3be93b6e0164ff4342e8421b411","Type":"ContainerStarted","Data":"fc38bf689f216725d47501000a0ee2463eb9ec9b0d4ea055aec3615ba48d41a5"} Jul 09 18:45:37.659675 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:37.659631 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fvbjs" podStartSLOduration=3.516076883 podStartE2EDuration="23.65961458s" totalImagesPullingTime="20.143537697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:37.659079525 +0000 UTC m=+24.160038588" watchObservedRunningTime="2026-07-09 18:45:37.65961458 +0000 UTC m=+24.160573618" Jul 09 18:45:37.692975 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:37.692920 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hjd48" podStartSLOduration=3.563151899 podStartE2EDuration="23.692902162s" totalImagesPullingTime="20.129750263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:37.678508393 +0000 UTC m=+24.179467431" watchObservedRunningTime="2026-07-09 18:45:37.692902162 +0000 UTC m=+24.193861192" Jul 09 18:45:37.693528 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:37.693505 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-249.ec2.internal" podStartSLOduration=21.693499533 podStartE2EDuration="21.693499533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:37.692844417 +0000 UTC m=+24.193803455" watchObservedRunningTime="2026-07-09 18:45:37.693499533 +0000 UTC m=+24.194458571" Jul 09 18:45:38.060032 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.059739 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:38.060140 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.059739 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:38.060197 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:38.060137 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:38.060197 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.059818 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:38.060197 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:38.060187 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:38.060310 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:38.060248 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:38.647920 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.647877 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" event={"ID":"92cf6223-3b85-4530-b4cd-7b2cf230a717","Type":"ContainerStarted","Data":"9cf35fdd4bfb5513e505b0164ce8a0b0538ebaad7d515a24ea09b8530e10ad09"} Jul 09 18:45:38.649610 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.649584 2587 generic.go:356] "Generic (PLEG): container finished" podID="226d8d2c98aa25e6e0873bbabeb7b26f" containerID="fae585bb17653a688ab3a6d30f4be6bcd3de2c49e12328fe630d163e4e5debd5" exitCode=0 Jul 09 18:45:38.649721 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.649620 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" event={"ID":"226d8d2c98aa25e6e0873bbabeb7b26f","Type":"ContainerDied","Data":"fae585bb17653a688ab3a6d30f4be6bcd3de2c49e12328fe630d163e4e5debd5"} Jul 09 18:45:38.651370 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.651301 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bg9n2" event={"ID":"dc77411f-12b3-40e2-b71a-b338eabd5065","Type":"ContainerStarted","Data":"b1de432f3b8ab5a8bdfb6ebc85ddf5d4fa38e0c9e332552f06dedfde26698c82"} Jul 09 18:45:38.652855 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.652832 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lfllq" event={"ID":"b7eaf8ff-494a-4a80-a065-77b2b7a1c78f","Type":"ContainerStarted","Data":"1ccad4e293a715825101b087b6ccb383dce1d9cfef50fe479b3a852877d1a043"} Jul 09 18:45:38.654692 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.654643 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r2x85" event={"ID":"7fb5d87d-2207-4215-900e-9f5cbde935de","Type":"ContainerStarted","Data":"646a0e264b53d9c6cdfdbe9503a66c80b2142920f43c8073102f2cbd41368627"} Jul 09 18:45:38.656311 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.656292 2587 generic.go:356] "Generic (PLEG): container finished" podID="adfe3fbd-54e0-4772-8240-4afdd653ac76" containerID="8b1fda8b5f156e36ef7efbe5940b309691d83581b7c2b16f08698b02598de481" exitCode=0 Jul 09 18:45:38.656391 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.656359 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerDied","Data":"8b1fda8b5f156e36ef7efbe5940b309691d83581b7c2b16f08698b02598de481"} Jul 09 18:45:38.658180 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.658107 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dh7hx" event={"ID":"542ce291-5b21-4361-b4a1-58e67e29c530","Type":"ContainerStarted","Data":"12e35b22111457cb4dac832aa8b415514b6982df778b9681f6af4ba892e071d5"} Jul 09 18:45:38.661164 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.661145 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:45:38.661524 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.661503 2587 generic.go:356] "Generic (PLEG): container finished" podID="f1b9b361-c850-40cd-af98-cd00992eb6d3" containerID="850e481befc82f1db581d2a0d6661e1648ac4a8508aa8600adccd383fd60fe73" exitCode=1 Jul 09 18:45:38.661613 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.661583 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"1717afe8479b25306fa6d45009a5a0c7bc7dcf75eadc5967877bb190435d2284"} Jul 09 18:45:38.661671 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.661645 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"da476edadd05ec901fed811a1883f44d55408546f5f27cc4d1fedddba0406811"} Jul 09 18:45:38.661725 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.661678 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"f9aec9f2a996ef3a9a0f47f84c9d5847a3275131209fbd3c78835ec9ec2dc8ad"} Jul 09 18:45:38.661775 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.661723 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"d5fb79801aaef165b9782e4c3cc96c0a76cda32b2ab46174153565dc9bed3132"} Jul 09 18:45:38.661821 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.661775 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerDied","Data":"850e481befc82f1db581d2a0d6661e1648ac4a8508aa8600adccd383fd60fe73"} Jul 09 18:45:38.685397 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.685349 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lfllq" podStartSLOduration=4.574685657 podStartE2EDuration="24.685330496s" totalImagesPullingTime="20.110644839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:38.684752201 +0000 UTC m=+25.185711241" watchObservedRunningTime="2026-07-09 18:45:38.685330496 +0000 UTC m=+25.186289535" Jul 09 18:45:38.722338 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.722302 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dh7hx" podStartSLOduration=8.157324377 podStartE2EDuration="23.722287761s" totalImagesPullingTime="15.564963384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:38.700975066 +0000 UTC m=+25.201934131" watchObservedRunningTime="2026-07-09 18:45:38.722287761 +0000 UTC m=+25.223246858" Jul 09 18:45:38.738378 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.738341 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r2x85" podStartSLOduration=9.146330698 podStartE2EDuration="24.738326966s" totalImagesPullingTime="15.591996268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:38.737963842 +0000 UTC m=+25.238922906" watchObservedRunningTime="2026-07-09 18:45:38.738326966 +0000 UTC m=+25.239286003" Jul 09 18:45:38.756169 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:38.756132 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bg9n2" podStartSLOduration=13.03774992 podStartE2EDuration="24.756119719s" totalImagesPullingTime="11.718369799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:38.756053331 +0000 UTC m=+25.257012369" watchObservedRunningTime="2026-07-09 18:45:38.756119719 +0000 UTC m=+25.257078757" Jul 09 18:45:39.545640 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:39.545616 2587 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 18:45:39.666705 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:39.666619 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" event={"ID":"92cf6223-3b85-4530-b4cd-7b2cf230a717","Type":"ContainerStarted","Data":"c066170436585fa46c990b9dc44c95d312d3b026edd8783ae9c295bc5df225e8"} Jul 09 18:45:39.668506 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:39.668196 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:39.668642 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:39.668554 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" event={"ID":"226d8d2c98aa25e6e0873bbabeb7b26f","Type":"ContainerStarted","Data":"38bf90f9c8765158b434506e66be05bc7137947fcaa4e6d801fb614d54d3eff0"} Jul 09 18:45:39.670648 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:39.670193 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:39.686731 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:39.686686 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-249.ec2.internal" podStartSLOduration=23.6866736 podStartE2EDuration="23.6866736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:39.686388484 +0000 UTC m=+26.187347524" watchObservedRunningTime="2026-07-09 18:45:39.6866736 +0000 UTC m=+26.187632638" Jul 09 18:45:40.060249 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.060097 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:40.060249 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.060144 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:40.060249 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:40.060235 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:40.060603 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.060348 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:40.060603 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:40.060465 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:40.060603 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:40.060536 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:40.498224 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.498130 2587 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:45:39.545636425Z","UUID":"98203c03-f2a5-4ed6-8524-193dba794019","Handler":null,"Name":"","Endpoint":""} Jul 09 18:45:40.500544 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.500508 2587 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 18:45:40.500544 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.500546 2587 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 18:45:40.675627 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.675559 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:45:40.676741 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.676710 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"361bc236dcc39d085e23ea9ab11d8082909cb870cdc902bf072f90993a382126"} Jul 09 18:45:40.677241 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.677218 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:40.677745 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:40.677720 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bg9n2" Jul 09 18:45:41.680687 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:41.680488 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" event={"ID":"92cf6223-3b85-4530-b4cd-7b2cf230a717","Type":"ContainerStarted","Data":"2c0c4e53d2de0cd4f2203ac82af49066947a2122bc4f89e68e4818f51ad101b8"} Jul 09 18:45:41.700709 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:41.700659 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgd2g" podStartSLOduration=9.573921323 podStartE2EDuration="27.700640704s" totalImagesPullingTime="18.126719381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:41.700516022 +0000 UTC m=+28.201475061" watchObservedRunningTime="2026-07-09 18:45:41.700640704 +0000 UTC m=+28.201599745" Jul 09 18:45:42.059969 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:42.059875 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:42.060129 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:42.059875 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:42.060129 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:42.060019 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:42.060129 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:42.060007 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:42.060129 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:42.060119 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:42.060341 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:42.060209 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:43.686932 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.686753 2587 generic.go:356] "Generic (PLEG): container finished" podID="adfe3fbd-54e0-4772-8240-4afdd653ac76" containerID="86fb396e5c96aa25eaeaada04eb61616499ec59c9393cbf071468558bc90e2bc" exitCode=0 Jul 09 18:45:43.687718 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.686812 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerDied","Data":"86fb396e5c96aa25eaeaada04eb61616499ec59c9393cbf071468558bc90e2bc"} Jul 09 18:45:43.690460 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.690444 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:45:43.690781 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.690762 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"6a1a421bc2ac029690a3fe8cbb64c0be7b301d48024764ec24298ea0a4f380e9"} Jul 09 18:45:43.691027 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.691006 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:43.691078 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.691057 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:43.691231 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.691213 2587 scope.go:117] "RemoveContainer" containerID="850e481befc82f1db581d2a0d6661e1648ac4a8508aa8600adccd383fd60fe73" Jul 09 18:45:43.707450 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:43.707433 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:44.060043 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.059968 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:44.060238 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.060055 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:44.060238 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:44.060060 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:44.060238 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:44.060134 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:44.060238 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.060159 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:44.060238 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:44.060211 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:44.696539 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.696514 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:45:44.697026 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.696865 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" event={"ID":"f1b9b361-c850-40cd-af98-cd00992eb6d3","Type":"ContainerStarted","Data":"6185a487644670bf5b35834196fcad17874c1b4700424e6e0110d6b0518511df"} Jul 09 18:45:44.697096 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.697079 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:44.710998 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.710978 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:45:44.739741 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:44.739699 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" podStartSLOduration=10.123420237 podStartE2EDuration="30.739686208s" totalImagesPullingTime="20.616265971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:44.738294036 +0000 UTC m=+31.239253094" watchObservedRunningTime="2026-07-09 18:45:44.739686208 +0000 UTC m=+31.240645245" Jul 09 18:45:45.277826 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.277617 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b62dq"] Jul 09 18:45:45.277997 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.277861 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:45.277997 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:45.277961 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:45.280557 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.280530 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5vkd"] Jul 09 18:45:45.280683 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.280619 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:45.280749 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:45.280718 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:45.281251 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.281220 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vf5zs"] Jul 09 18:45:45.281397 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.281333 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:45.281462 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:45.281433 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:45.701330 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.701299 2587 generic.go:356] "Generic (PLEG): container finished" podID="adfe3fbd-54e0-4772-8240-4afdd653ac76" containerID="42780f339b4832b53ea57b690a9f5a5dabbf3a6d28e304b5356d62704c950904" exitCode=0 Jul 09 18:45:45.701765 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:45.701362 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerDied","Data":"42780f339b4832b53ea57b690a9f5a5dabbf3a6d28e304b5356d62704c950904"} Jul 09 18:45:47.060128 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:47.060100 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:47.060585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:47.060206 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:47.060585 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:47.060206 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:47.060585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:47.060214 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:47.060585 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:47.060330 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:47.060585 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:47.060384 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:47.708098 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:47.708062 2587 generic.go:356] "Generic (PLEG): container finished" podID="adfe3fbd-54e0-4772-8240-4afdd653ac76" containerID="1aff441332bc34b4c2d5828e8df54b70939c54b2d3c3bfa50ab7924d1f18557e" exitCode=0 Jul 09 18:45:47.708245 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:47.708109 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerDied","Data":"1aff441332bc34b4c2d5828e8df54b70939c54b2d3c3bfa50ab7924d1f18557e"} Jul 09 18:45:48.346155 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:48.346127 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:48.346623 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:48.346179 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:48.346623 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.346270 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:48.346623 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.346275 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:48.346623 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.346318 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret podName:88545bfb-fc18-4f99-ab53-894af89974a2 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:20.34630508 +0000 UTC m=+66.847264115 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret") pod "global-pull-secret-syncer-vf5zs" (UID: "88545bfb-fc18-4f99-ab53-894af89974a2") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:45:48.346623 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.346330 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs podName:f30cc372-130d-417c-a1c4-2754e9d9f80d nodeName:}" failed. No retries permitted until 2026-07-09 18:46:20.346324176 +0000 UTC m=+66.847283195 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs") pod "network-metrics-daemon-b62dq" (UID: "f30cc372-130d-417c-a1c4-2754e9d9f80d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:45:48.447305 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:48.447276 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:48.447462 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.447385 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:45:48.447462 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.447400 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:45:48.447462 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.447408 2587 projected.go:196] Error preparing data for projected volume kube-api-access-mcfpz for pod openshift-network-diagnostics/network-check-target-l5vkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:48.447462 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:48.447454 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz podName:fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:20.447442066 +0000 UTC m=+66.948401083 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcfpz" (UniqueName: "kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz") pod "network-check-target-l5vkd" (UID: "fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:45:49.059235 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:49.059202 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:49.059412 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:49.059252 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:49.059412 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:49.059329 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:49.059412 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:49.059391 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:49.059579 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:49.059508 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:49.059579 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:49.059569 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:51.059824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.059745 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:51.060424 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.059853 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:51.060424 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:51.059866 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vf5zs" podUID="88545bfb-fc18-4f99-ab53-894af89974a2" Jul 09 18:45:51.060424 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.059888 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:51.060424 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:51.059987 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5vkd" podUID="fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4" Jul 09 18:45:51.060424 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:51.060078 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62dq" podUID="f30cc372-130d-417c-a1c4-2754e9d9f80d" Jul 09 18:45:51.241883 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.241853 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-249.ec2.internal" event="NodeReady" Jul 09 18:45:51.242065 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.242011 2587 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 18:45:51.292412 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.292384 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9p7tm"] Jul 09 18:45:51.295091 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.295070 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.295401 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.295379 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wlxp4"] Jul 09 18:45:51.296954 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.296929 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.298327 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.298307 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 18:45:51.298557 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.298542 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5t98t\"" Jul 09 18:45:51.298850 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.298832 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 18:45:51.299515 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.299492 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 18:45:51.299647 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.299624 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9zdkl\"" Jul 09 18:45:51.299717 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.299626 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 18:45:51.299717 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.299689 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 18:45:51.307578 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.307552 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9p7tm"] Jul 09 18:45:51.310557 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.310508 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wlxp4"] Jul 09 18:45:51.370180 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.370142 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zzkj\" (UniqueName: \"kubernetes.io/projected/949afb45-35a9-4750-99a8-9bf32d8f8e25-kube-api-access-5zzkj\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.370328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.370248 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.370328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.370288 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fcf474d-2db5-4a19-803c-414e4ab64a2b-cert\") pod \"ingress-canary-wlxp4\" (UID: \"6fcf474d-2db5-4a19-803c-414e4ab64a2b\") " pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.370328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.370315 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkvlg\" (UniqueName: \"kubernetes.io/projected/6fcf474d-2db5-4a19-803c-414e4ab64a2b-kube-api-access-rkvlg\") pod \"ingress-canary-wlxp4\" (UID: \"6fcf474d-2db5-4a19-803c-414e4ab64a2b\") " pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.370440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.370343 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/949afb45-35a9-4750-99a8-9bf32d8f8e25-config-volume\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.370440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.370362 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/949afb45-35a9-4750-99a8-9bf32d8f8e25-tmp-dir\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.470793 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.470758 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fcf474d-2db5-4a19-803c-414e4ab64a2b-cert\") pod \"ingress-canary-wlxp4\" (UID: \"6fcf474d-2db5-4a19-803c-414e4ab64a2b\") " pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.470793 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.470802 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkvlg\" (UniqueName: \"kubernetes.io/projected/6fcf474d-2db5-4a19-803c-414e4ab64a2b-kube-api-access-rkvlg\") pod \"ingress-canary-wlxp4\" (UID: \"6fcf474d-2db5-4a19-803c-414e4ab64a2b\") " pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.471050 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.470834 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/949afb45-35a9-4750-99a8-9bf32d8f8e25-config-volume\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.471050 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.470999 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/949afb45-35a9-4750-99a8-9bf32d8f8e25-tmp-dir\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.471183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.471062 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zzkj\" (UniqueName: \"kubernetes.io/projected/949afb45-35a9-4750-99a8-9bf32d8f8e25-kube-api-access-5zzkj\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.471183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.471153 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.471288 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:51.471266 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:45:51.471344 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:51.471335 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls podName:949afb45-35a9-4750-99a8-9bf32d8f8e25 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:51.97131712 +0000 UTC m=+38.472276157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls") pod "dns-default-9p7tm" (UID: "949afb45-35a9-4750-99a8-9bf32d8f8e25") : secret "dns-default-metrics-tls" not found Jul 09 18:45:51.471448 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.471426 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/949afb45-35a9-4750-99a8-9bf32d8f8e25-config-volume\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.471448 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.471442 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/949afb45-35a9-4750-99a8-9bf32d8f8e25-tmp-dir\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.475395 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.475376 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fcf474d-2db5-4a19-803c-414e4ab64a2b-cert\") pod \"ingress-canary-wlxp4\" (UID: \"6fcf474d-2db5-4a19-803c-414e4ab64a2b\") " pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.481835 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.481812 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkvlg\" (UniqueName: \"kubernetes.io/projected/6fcf474d-2db5-4a19-803c-414e4ab64a2b-kube-api-access-rkvlg\") pod \"ingress-canary-wlxp4\" (UID: \"6fcf474d-2db5-4a19-803c-414e4ab64a2b\") " pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.484146 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.484128 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zzkj\" (UniqueName: \"kubernetes.io/projected/949afb45-35a9-4750-99a8-9bf32d8f8e25-kube-api-access-5zzkj\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.614364 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.614280 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wlxp4" Jul 09 18:45:51.975330 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.975144 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:51.975439 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:51.975283 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:45:51.975479 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:51.975452 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls podName:949afb45-35a9-4750-99a8-9bf32d8f8e25 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:52.975437917 +0000 UTC m=+39.476396933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls") pod "dns-default-9p7tm" (UID: "949afb45-35a9-4750-99a8-9bf32d8f8e25") : secret "dns-default-metrics-tls" not found Jul 09 18:45:51.993498 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:51.993471 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wlxp4"] Jul 09 18:45:51.997337 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:51.997310 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fcf474d_2db5_4a19_803c_414e4ab64a2b.slice/crio-29e835f38feca348f65a6894a6236078dc7b41f1888b915b7b99b8eb566fd4e7 WatchSource:0}: Error finding container 29e835f38feca348f65a6894a6236078dc7b41f1888b915b7b99b8eb566fd4e7: Status 404 returned error can't find the container with id 29e835f38feca348f65a6894a6236078dc7b41f1888b915b7b99b8eb566fd4e7 Jul 09 18:45:52.721779 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:52.721733 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wlxp4" event={"ID":"6fcf474d-2db5-4a19-803c-414e4ab64a2b","Type":"ContainerStarted","Data":"29e835f38feca348f65a6894a6236078dc7b41f1888b915b7b99b8eb566fd4e7"} Jul 09 18:45:52.724825 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:52.724795 2587 generic.go:356] "Generic (PLEG): container finished" podID="adfe3fbd-54e0-4772-8240-4afdd653ac76" containerID="ca3c5d72ac5bf54c13d1998f6957b809d8366690aef5f963471a350cb1d36120" exitCode=0 Jul 09 18:45:52.724956 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:52.724840 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerDied","Data":"ca3c5d72ac5bf54c13d1998f6957b809d8366690aef5f963471a350cb1d36120"} Jul 09 18:45:52.982143 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:52.982053 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:52.982306 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:52.982236 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:45:52.982353 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:52.982312 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls podName:949afb45-35a9-4750-99a8-9bf32d8f8e25 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:54.982293136 +0000 UTC m=+41.483252179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls") pod "dns-default-9p7tm" (UID: "949afb45-35a9-4750-99a8-9bf32d8f8e25") : secret "dns-default-metrics-tls" not found Jul 09 18:45:53.060191 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.060151 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:45:53.060378 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.060152 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:45:53.060378 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.060292 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:45:53.063695 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.063320 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:45:53.063695 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.063376 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 18:45:53.063695 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.063401 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-prgmc\"" Jul 09 18:45:53.063695 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.063494 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:45:53.063695 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.063332 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 18:45:53.063695 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.063639 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ckpt2\"" Jul 09 18:45:53.212776 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.212742 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9"] Jul 09 18:45:53.214655 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.214631 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" Jul 09 18:45:53.217486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.217461 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl"] Jul 09 18:45:53.219336 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.219293 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5r4rb"] Jul 09 18:45:53.219458 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.219393 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.219648 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.219616 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:53.219753 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.219655 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:53.220136 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.220113 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g24dz\"" Jul 09 18:45:53.221126 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.221108 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-8g5bp"] Jul 09 18:45:53.221260 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.221244 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.222642 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.222622 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:53.222731 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.222703 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-z6vjg\"" Jul 09 18:45:53.222779 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.222741 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.223127 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.223104 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 18:45:53.223351 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.223332 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:53.223457 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.223438 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 18:45:53.224149 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.224040 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 18:45:53.224682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.224439 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 18:45:53.224682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.224453 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-cb7mg\"" Jul 09 18:45:53.224682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.224592 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 18:45:53.224860 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.224772 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 18:45:53.225383 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.225368 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-z9rh9\"" Jul 09 18:45:53.226064 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.226044 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 18:45:53.227592 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.227482 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:53.227711 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.227693 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 18:45:53.229005 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.228985 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:53.239498 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.238368 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9"] Jul 09 18:45:53.240881 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.240678 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 18:45:53.240881 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.240836 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5r4rb"] Jul 09 18:45:53.240881 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.240847 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 18:45:53.241830 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.241812 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-8g5bp"] Jul 09 18:45:53.260104 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.260082 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl"] Jul 09 18:45:53.284059 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284031 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzhsl\" (UniqueName: \"kubernetes.io/projected/7ae20ec9-71d1-44c8-a486-040683dfa34f-kube-api-access-jzhsl\") pod \"volume-data-source-validator-78495597f6-4r8w9\" (UID: \"7ae20ec9-71d1-44c8-a486-040683dfa34f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" Jul 09 18:45:53.284207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284069 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5400ba01-ed2f-40e4-8c04-85be98f44d1a-tmp\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.284207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284090 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krv6s\" (UniqueName: \"kubernetes.io/projected/5400ba01-ed2f-40e4-8c04-85be98f44d1a-kube-api-access-krv6s\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.284207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284108 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxd24\" (UniqueName: \"kubernetes.io/projected/de304309-203f-46ba-ba4d-024e0ab59b8c-kube-api-access-hxd24\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.284207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284160 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5400ba01-ed2f-40e4-8c04-85be98f44d1a-serving-cert\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.284417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284212 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt52k\" (UniqueName: \"kubernetes.io/projected/6948c276-c69c-4349-bff2-cdec236e5419-kube-api-access-tt52k\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.284417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284243 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5400ba01-ed2f-40e4-8c04-85be98f44d1a-snapshots\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.284417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284258 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de304309-203f-46ba-ba4d-024e0ab59b8c-trusted-ca\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.284417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284304 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6948c276-c69c-4349-bff2-cdec236e5419-config\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.284417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284323 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5400ba01-ed2f-40e4-8c04-85be98f44d1a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.284417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284370 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6948c276-c69c-4349-bff2-cdec236e5419-serving-cert\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.284417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284388 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de304309-203f-46ba-ba4d-024e0ab59b8c-serving-cert\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.284749 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284432 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5400ba01-ed2f-40e4-8c04-85be98f44d1a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.284749 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.284462 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de304309-203f-46ba-ba4d-024e0ab59b8c-config\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.320300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.320267 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz"] Jul 09 18:45:53.322629 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.322607 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:53.325082 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.325061 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph"] Jul 09 18:45:53.325907 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.325722 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 18:45:53.326684 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.326499 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:53.326684 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.326552 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:53.326684 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.326597 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-srmbg\"" Jul 09 18:45:53.327083 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.327057 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.329616 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.329598 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:45:53.329711 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.329678 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qkf7m\"" Jul 09 18:45:53.329839 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.329823 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 09 18:45:53.329907 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.329868 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 09 18:45:53.330203 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.330182 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 09 18:45:53.335840 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.335820 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz"] Jul 09 18:45:53.338727 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.338710 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph"] Jul 09 18:45:53.385183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385151 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:53.385326 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385193 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sw75\" (UniqueName: \"kubernetes.io/projected/cd5de19d-1436-4636-9660-008223b3a68e-kube-api-access-8sw75\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:53.385326 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385258 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84b890da-cc0c-4274-9b59-7839dad6a083-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.385326 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385292 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt52k\" (UniqueName: \"kubernetes.io/projected/6948c276-c69c-4349-bff2-cdec236e5419-kube-api-access-tt52k\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.385491 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385333 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5400ba01-ed2f-40e4-8c04-85be98f44d1a-snapshots\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.385491 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385360 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de304309-203f-46ba-ba4d-024e0ab59b8c-trusted-ca\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.385491 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385424 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6948c276-c69c-4349-bff2-cdec236e5419-config\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.385491 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385458 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5400ba01-ed2f-40e4-8c04-85be98f44d1a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.385682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385491 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6948c276-c69c-4349-bff2-cdec236e5419-serving-cert\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.385682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385521 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b890da-cc0c-4274-9b59-7839dad6a083-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.385682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385549 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de304309-203f-46ba-ba4d-024e0ab59b8c-serving-cert\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.385682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385600 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5400ba01-ed2f-40e4-8c04-85be98f44d1a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.385682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385623 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de304309-203f-46ba-ba4d-024e0ab59b8c-config\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.385682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385648 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89mh6\" (UniqueName: \"kubernetes.io/projected/84b890da-cc0c-4274-9b59-7839dad6a083-kube-api-access-89mh6\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.385682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385678 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzhsl\" (UniqueName: \"kubernetes.io/projected/7ae20ec9-71d1-44c8-a486-040683dfa34f-kube-api-access-jzhsl\") pod \"volume-data-source-validator-78495597f6-4r8w9\" (UID: \"7ae20ec9-71d1-44c8-a486-040683dfa34f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" Jul 09 18:45:53.386074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385702 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5400ba01-ed2f-40e4-8c04-85be98f44d1a-tmp\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.386074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385725 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krv6s\" (UniqueName: \"kubernetes.io/projected/5400ba01-ed2f-40e4-8c04-85be98f44d1a-kube-api-access-krv6s\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.386074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385751 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxd24\" (UniqueName: \"kubernetes.io/projected/de304309-203f-46ba-ba4d-024e0ab59b8c-kube-api-access-hxd24\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.386074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.385796 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5400ba01-ed2f-40e4-8c04-85be98f44d1a-serving-cert\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.386074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.386033 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5400ba01-ed2f-40e4-8c04-85be98f44d1a-snapshots\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.386301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.386103 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6948c276-c69c-4349-bff2-cdec236e5419-config\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.386301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.386180 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5400ba01-ed2f-40e4-8c04-85be98f44d1a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.386301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.386295 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de304309-203f-46ba-ba4d-024e0ab59b8c-trusted-ca\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.386451 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.386309 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5400ba01-ed2f-40e4-8c04-85be98f44d1a-tmp\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.386903 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.386863 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de304309-203f-46ba-ba4d-024e0ab59b8c-config\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.387112 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.387068 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5400ba01-ed2f-40e4-8c04-85be98f44d1a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.389636 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.389617 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6948c276-c69c-4349-bff2-cdec236e5419-serving-cert\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.390059 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.390036 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5400ba01-ed2f-40e4-8c04-85be98f44d1a-serving-cert\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.390128 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.390036 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de304309-203f-46ba-ba4d-024e0ab59b8c-serving-cert\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.397648 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.397589 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxd24\" (UniqueName: \"kubernetes.io/projected/de304309-203f-46ba-ba4d-024e0ab59b8c-kube-api-access-hxd24\") pod \"console-operator-7866497494-8g5bp\" (UID: \"de304309-203f-46ba-ba4d-024e0ab59b8c\") " pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.398873 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.398829 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krv6s\" (UniqueName: \"kubernetes.io/projected/5400ba01-ed2f-40e4-8c04-85be98f44d1a-kube-api-access-krv6s\") pod \"insights-operator-68d9dd94cf-5r4rb\" (UID: \"5400ba01-ed2f-40e4-8c04-85be98f44d1a\") " pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.399578 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.399524 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzhsl\" (UniqueName: \"kubernetes.io/projected/7ae20ec9-71d1-44c8-a486-040683dfa34f-kube-api-access-jzhsl\") pod \"volume-data-source-validator-78495597f6-4r8w9\" (UID: \"7ae20ec9-71d1-44c8-a486-040683dfa34f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" Jul 09 18:45:53.400322 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.400300 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt52k\" (UniqueName: \"kubernetes.io/projected/6948c276-c69c-4349-bff2-cdec236e5419-kube-api-access-tt52k\") pod \"service-ca-operator-7695fc6c8c-77lvl\" (UID: \"6948c276-c69c-4349-bff2-cdec236e5419\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.486916 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.486884 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89mh6\" (UniqueName: \"kubernetes.io/projected/84b890da-cc0c-4274-9b59-7839dad6a083-kube-api-access-89mh6\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.487074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.486966 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:53.487074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.486992 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sw75\" (UniqueName: \"kubernetes.io/projected/cd5de19d-1436-4636-9660-008223b3a68e-kube-api-access-8sw75\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:53.487074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.487032 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84b890da-cc0c-4274-9b59-7839dad6a083-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.487245 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.487093 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b890da-cc0c-4274-9b59-7839dad6a083-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.487245 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:53.487200 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 18:45:53.487353 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:53.487280 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls podName:cd5de19d-1436-4636-9660-008223b3a68e nodeName:}" failed. No retries permitted until 2026-07-09 18:45:53.987260299 +0000 UTC m=+40.488219338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls") pod "cluster-samples-operator-65db858894-kwxmz" (UID: "cd5de19d-1436-4636-9660-008223b3a68e") : secret "samples-operator-tls" not found Jul 09 18:45:53.487612 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.487595 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b890da-cc0c-4274-9b59-7839dad6a083-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.489521 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.489459 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84b890da-cc0c-4274-9b59-7839dad6a083-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.501506 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.501486 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sw75\" (UniqueName: \"kubernetes.io/projected/cd5de19d-1436-4636-9660-008223b3a68e-kube-api-access-8sw75\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:53.501605 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.501480 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89mh6\" (UniqueName: \"kubernetes.io/projected/84b890da-cc0c-4274-9b59-7839dad6a083-kube-api-access-89mh6\") pod \"kube-storage-version-migrator-operator-799cb7f958-lj7ph\" (UID: \"84b890da-cc0c-4274-9b59-7839dad6a083\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.529459 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.529436 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" Jul 09 18:45:53.545260 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.545237 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" Jul 09 18:45:53.551490 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.551464 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" Jul 09 18:45:53.557515 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.557043 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:53.557515 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.557235 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:45:53.658693 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.657085 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" Jul 09 18:45:53.675227 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.674573 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:53.733999 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.733122 2587 generic.go:356] "Generic (PLEG): container finished" podID="adfe3fbd-54e0-4772-8240-4afdd653ac76" containerID="0de776df876eb2226e7270074b0899e5ceb3e5489b91cb5fc52b8bfc4a26c8db" exitCode=0 Jul 09 18:45:53.733999 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.733178 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerDied","Data":"0de776df876eb2226e7270074b0899e5ceb3e5489b91cb5fc52b8bfc4a26c8db"} Jul 09 18:45:53.740002 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.739360 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9"] Jul 09 18:45:53.751348 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:53.751313 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae20ec9_71d1_44c8_a486_040683dfa34f.slice/crio-1067755c63aca56fc9b30ce723059c0772efa9c91b74b891aa6906580e1b473b WatchSource:0}: Error finding container 1067755c63aca56fc9b30ce723059c0772efa9c91b74b891aa6906580e1b473b: Status 404 returned error can't find the container with id 1067755c63aca56fc9b30ce723059c0772efa9c91b74b891aa6906580e1b473b Jul 09 18:45:53.770026 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.769997 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-8g5bp"] Jul 09 18:45:53.771326 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:53.771296 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde304309_203f_46ba_ba4d_024e0ab59b8c.slice/crio-20efef78c62ea9f78b20a7eef472fabf40b74922e5515ed6bb87f305613ea3bb WatchSource:0}: Error finding container 20efef78c62ea9f78b20a7eef472fabf40b74922e5515ed6bb87f305613ea3bb: Status 404 returned error can't find the container with id 20efef78c62ea9f78b20a7eef472fabf40b74922e5515ed6bb87f305613ea3bb Jul 09 18:45:53.776329 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.776264 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5r4rb"] Jul 09 18:45:53.780665 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:53.780640 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5400ba01_ed2f_40e4_8c04_85be98f44d1a.slice/crio-a162899df0263895080b00407e457270858d763d37b68b8a4bf51f3c0334c618 WatchSource:0}: Error finding container a162899df0263895080b00407e457270858d763d37b68b8a4bf51f3c0334c618: Status 404 returned error can't find the container with id a162899df0263895080b00407e457270858d763d37b68b8a4bf51f3c0334c618 Jul 09 18:45:53.784208 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.784148 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl"] Jul 09 18:45:53.787963 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:53.787913 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6948c276_c69c_4349_bff2_cdec236e5419.slice/crio-582c7bcd77f48450489b619c60b9a03e38ed41f9529ec2b5029afba90ccba26b WatchSource:0}: Error finding container 582c7bcd77f48450489b619c60b9a03e38ed41f9529ec2b5029afba90ccba26b: Status 404 returned error can't find the container with id 582c7bcd77f48450489b619c60b9a03e38ed41f9529ec2b5029afba90ccba26b Jul 09 18:45:53.838970 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.838924 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph"] Jul 09 18:45:53.992008 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:53.991725 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:53.992008 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:53.991896 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 18:45:53.992210 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:53.992053 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls podName:cd5de19d-1436-4636-9660-008223b3a68e nodeName:}" failed. No retries permitted until 2026-07-09 18:45:54.992031118 +0000 UTC m=+41.492990137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls") pod "cluster-samples-operator-65db858894-kwxmz" (UID: "cd5de19d-1436-4636-9660-008223b3a68e") : secret "samples-operator-tls" not found Jul 09 18:45:54.088902 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.088863 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:54.161203 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.161163 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:54.542584 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:45:54.542554 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b890da_cc0c_4274_9b59_7839dad6a083.slice/crio-91d9554d57a2164f27abab5c2c499ffe31cac1b47b5c58129592a202ce4cb67b WatchSource:0}: Error finding container 91d9554d57a2164f27abab5c2c499ffe31cac1b47b5c58129592a202ce4cb67b: Status 404 returned error can't find the container with id 91d9554d57a2164f27abab5c2c499ffe31cac1b47b5c58129592a202ce4cb67b Jul 09 18:45:54.736635 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.736601 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8g5bp" event={"ID":"de304309-203f-46ba-ba4d-024e0ab59b8c","Type":"ContainerStarted","Data":"20efef78c62ea9f78b20a7eef472fabf40b74922e5515ed6bb87f305613ea3bb"} Jul 09 18:45:54.737607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.737582 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" event={"ID":"7ae20ec9-71d1-44c8-a486-040683dfa34f","Type":"ContainerStarted","Data":"1067755c63aca56fc9b30ce723059c0772efa9c91b74b891aa6906580e1b473b"} Jul 09 18:45:54.738520 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.738499 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" event={"ID":"84b890da-cc0c-4274-9b59-7839dad6a083","Type":"ContainerStarted","Data":"91d9554d57a2164f27abab5c2c499ffe31cac1b47b5c58129592a202ce4cb67b"} Jul 09 18:45:54.739560 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.739535 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" event={"ID":"5400ba01-ed2f-40e4-8c04-85be98f44d1a","Type":"ContainerStarted","Data":"a162899df0263895080b00407e457270858d763d37b68b8a4bf51f3c0334c618"} Jul 09 18:45:54.740436 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.740418 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" event={"ID":"6948c276-c69c-4349-bff2-cdec236e5419","Type":"ContainerStarted","Data":"582c7bcd77f48450489b619c60b9a03e38ed41f9529ec2b5029afba90ccba26b"} Jul 09 18:45:54.743471 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.743454 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" event={"ID":"adfe3fbd-54e0-4772-8240-4afdd653ac76","Type":"ContainerStarted","Data":"973858161f86661f887acc20192dfeca1b9ceb66b3313316bb66460879ec530c"} Jul 09 18:45:54.769572 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.769367 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j4rwj" podStartSLOduration=14.144051448 podStartE2EDuration="40.769351108s" totalImagesPullingTime="26.62529966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:54.768069531 +0000 UTC m=+41.269028584" watchObservedRunningTime="2026-07-09 18:45:54.769351108 +0000 UTC m=+41.270310148" Jul 09 18:45:54.898001 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.897975 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:54.964543 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:54.964052 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:55.000100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:55.000062 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:55.000252 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:55.000118 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:55.000252 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:55.000238 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 18:45:55.000377 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:55.000270 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:45:55.000377 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:55.000321 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls podName:cd5de19d-1436-4636-9660-008223b3a68e nodeName:}" failed. No retries permitted until 2026-07-09 18:45:57.00030034 +0000 UTC m=+43.501259371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls") pod "cluster-samples-operator-65db858894-kwxmz" (UID: "cd5de19d-1436-4636-9660-008223b3a68e") : secret "samples-operator-tls" not found Jul 09 18:45:55.000377 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:55.000342 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls podName:949afb45-35a9-4750-99a8-9bf32d8f8e25 nodeName:}" failed. No retries permitted until 2026-07-09 18:45:59.000332023 +0000 UTC m=+45.501291066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls") pod "dns-default-9p7tm" (UID: "949afb45-35a9-4750-99a8-9bf32d8f8e25") : secret "dns-default-metrics-tls" not found Jul 09 18:45:55.042222 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:55.042100 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:55.143383 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:55.143304 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:55.750564 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:55.749542 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wlxp4" event={"ID":"6fcf474d-2db5-4a19-803c-414e4ab64a2b","Type":"ContainerStarted","Data":"25ad603e53ae20d6d687adba164ba8cee671f2b066f899e28ab007279f4fece2"} Jul 09 18:45:55.776564 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:55.776099 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wlxp4" podStartSLOduration=1.908050909 podStartE2EDuration="4.776078535s" totalImagesPullingTime="2.868027626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:55.772379008 +0000 UTC m=+42.273338047" watchObservedRunningTime="2026-07-09 18:45:55.776078535 +0000 UTC m=+42.277037576" Jul 09 18:45:57.017931 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:57.017897 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:45:57.018367 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:57.018043 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 18:45:57.018367 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:57.018100 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls podName:cd5de19d-1436-4636-9660-008223b3a68e nodeName:}" failed. No retries permitted until 2026-07-09 18:46:01.018086626 +0000 UTC m=+47.519045647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls") pod "cluster-samples-operator-65db858894-kwxmz" (UID: "cd5de19d-1436-4636-9660-008223b3a68e") : secret "samples-operator-tls" not found Jul 09 18:45:59.038668 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.038634 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:45:59.039046 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:59.038750 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:45:59.039046 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:45:59.038805 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls podName:949afb45-35a9-4750-99a8-9bf32d8f8e25 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:07.038790087 +0000 UTC m=+53.539749122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls") pod "dns-default-9p7tm" (UID: "949afb45-35a9-4750-99a8-9bf32d8f8e25") : secret "dns-default-metrics-tls" not found Jul 09 18:45:59.525718 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.525693 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:59.630009 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.629936 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:45:59.762678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.762580 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" event={"ID":"6948c276-c69c-4349-bff2-cdec236e5419","Type":"ContainerStarted","Data":"368f048c32c20aac78c808a429f8e644facbd56008b2947355509171352596f1"} Jul 09 18:45:59.764365 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.764342 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/0.log" Jul 09 18:45:59.764481 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.764384 2587 generic.go:356] "Generic (PLEG): container finished" podID="de304309-203f-46ba-ba4d-024e0ab59b8c" containerID="dcc42c339380882a6810ab4ef418ed613f908a5905037b2a7aa0e364c67c838b" exitCode=255 Jul 09 18:45:59.764481 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.764455 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8g5bp" event={"ID":"de304309-203f-46ba-ba4d-024e0ab59b8c","Type":"ContainerDied","Data":"dcc42c339380882a6810ab4ef418ed613f908a5905037b2a7aa0e364c67c838b"} Jul 09 18:45:59.764676 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.764644 2587 scope.go:117] "RemoveContainer" containerID="dcc42c339380882a6810ab4ef418ed613f908a5905037b2a7aa0e364c67c838b" Jul 09 18:45:59.766300 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.766274 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" event={"ID":"7ae20ec9-71d1-44c8-a486-040683dfa34f","Type":"ContainerStarted","Data":"d9f26a3c242c0d4701f8cdbfee9303b9cccd6a81bb0e0e5bd5162d818cbf7bf7"} Jul 09 18:45:59.768487 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.768465 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" event={"ID":"84b890da-cc0c-4274-9b59-7839dad6a083","Type":"ContainerStarted","Data":"0ff1e3d58a578f03d8095778561344e7b9d1d6203dce0b6210f668f4d3e5228f"} Jul 09 18:45:59.770016 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.769996 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" event={"ID":"5400ba01-ed2f-40e4-8c04-85be98f44d1a","Type":"ContainerStarted","Data":"5137d4c7949dcd4396ed82f7131f287adc0a9215a07e9f26d713d7f324e315aa"} Jul 09 18:45:59.780392 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.780347 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" podStartSLOduration=1.046730342 podStartE2EDuration="6.780331184s" totalImagesPullingTime="5.733600842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:59.779893946 +0000 UTC m=+46.280852984" watchObservedRunningTime="2026-07-09 18:45:59.780331184 +0000 UTC m=+46.281290222" Jul 09 18:45:59.797867 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.797820 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" podStartSLOduration=1.053088713 podStartE2EDuration="6.79780304s" totalImagesPullingTime="5.744714327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:59.796354177 +0000 UTC m=+46.297313220" watchObservedRunningTime="2026-07-09 18:45:59.79780304 +0000 UTC m=+46.298762082" Jul 09 18:45:59.836059 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.836010 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4r8w9" podStartSLOduration=1.11424193 podStartE2EDuration="6.835991352s" totalImagesPullingTime="5.721749422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:59.834679143 +0000 UTC m=+46.335638179" watchObservedRunningTime="2026-07-09 18:45:59.835991352 +0000 UTC m=+46.336950391" Jul 09 18:45:59.852661 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.852615 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" podStartSLOduration=1.872283219 podStartE2EDuration="6.852598243s" totalImagesPullingTime="4.980315024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:59.851669745 +0000 UTC m=+46.352628775" watchObservedRunningTime="2026-07-09 18:45:59.852598243 +0000 UTC m=+46.353557282" Jul 09 18:45:59.926378 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:45:59.926351 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:00.031440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.027793 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:00.557975 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.557932 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:00.655042 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.655009 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:00.780021 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.779992 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/1.log" Jul 09 18:46:00.780516 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.780495 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/0.log" Jul 09 18:46:00.780619 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.780538 2587 generic.go:356] "Generic (PLEG): container finished" podID="de304309-203f-46ba-ba4d-024e0ab59b8c" containerID="15fd6348756ee0684b33d1b5f5a797344123bf0955411e963d46b358de6c9883" exitCode=255 Jul 09 18:46:00.780681 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.780642 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8g5bp" event={"ID":"de304309-203f-46ba-ba4d-024e0ab59b8c","Type":"ContainerDied","Data":"15fd6348756ee0684b33d1b5f5a797344123bf0955411e963d46b358de6c9883"} Jul 09 18:46:00.780735 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.780703 2587 scope.go:117] "RemoveContainer" containerID="dcc42c339380882a6810ab4ef418ed613f908a5905037b2a7aa0e364c67c838b" Jul 09 18:46:00.781074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.781050 2587 scope.go:117] "RemoveContainer" containerID="15fd6348756ee0684b33d1b5f5a797344123bf0955411e963d46b358de6c9883" Jul 09 18:46:00.781254 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:00.781231 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-8g5bp_openshift-console-operator(de304309-203f-46ba-ba4d-024e0ab59b8c)\"" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podUID="de304309-203f-46ba-ba4d-024e0ab59b8c" Jul 09 18:46:00.783848 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.783825 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6wkrq"] Jul 09 18:46:00.801902 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.801880 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6wkrq"] Jul 09 18:46:00.802026 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.802012 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.804990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.804756 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 18:46:00.804990 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.804810 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 18:46:00.805142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.804996 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qssxl\"" Jul 09 18:46:00.825399 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.825371 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:00.853906 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.853873 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/026a8fa5-9e41-4153-98f4-439bb0f833a9-crio-socket\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.854050 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.853927 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/026a8fa5-9e41-4153-98f4-439bb0f833a9-data-volume\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.854050 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.854003 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.854263 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.854242 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.854871 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.854364 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5s8j\" (UniqueName: \"kubernetes.io/projected/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-api-access-s5s8j\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.919126 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.919075 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:00.955107 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.955084 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/026a8fa5-9e41-4153-98f4-439bb0f833a9-crio-socket\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.955210 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.955110 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/026a8fa5-9e41-4153-98f4-439bb0f833a9-data-volume\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.955210 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.955133 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.955210 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.955161 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.955210 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:00.955208 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:46:00.955416 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:00.955264 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:01.455248064 +0000 UTC m=+47.956207081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : configmap "kube-rbac-proxy" not found Jul 09 18:46:00.955416 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.955276 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5s8j\" (UniqueName: \"kubernetes.io/projected/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-api-access-s5s8j\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.955416 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.955326 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/026a8fa5-9e41-4153-98f4-439bb0f833a9-crio-socket\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.955416 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:00.955338 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:46:00.955416 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:00.955391 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:01.455375664 +0000 UTC m=+47.956334682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : secret "insights-runtime-extractor-tls" not found Jul 09 18:46:00.955609 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.955461 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/026a8fa5-9e41-4153-98f4-439bb0f833a9-data-volume\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:00.968044 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:00.968019 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5s8j\" (UniqueName: \"kubernetes.io/projected/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-api-access-s5s8j\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:01.056521 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.056495 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:46:01.056684 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:01.056662 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 18:46:01.056747 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:01.056737 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls podName:cd5de19d-1436-4636-9660-008223b3a68e nodeName:}" failed. No retries permitted until 2026-07-09 18:46:09.056718247 +0000 UTC m=+55.557677266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls") pod "cluster-samples-operator-65db858894-kwxmz" (UID: "cd5de19d-1436-4636-9660-008223b3a68e") : secret "samples-operator-tls" not found Jul 09 18:46:01.329800 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.329768 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:01.403299 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.403259 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:01.459792 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.459759 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:01.459974 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.459799 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:01.459974 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:01.459878 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:46:01.459974 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:01.459926 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:46:01.459974 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:01.459972 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:02.459938254 +0000 UTC m=+48.960897274 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : configmap "kube-rbac-proxy" not found Jul 09 18:46:01.460128 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:01.459988 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:02.459981442 +0000 UTC m=+48.960940461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : secret "insights-runtime-extractor-tls" not found Jul 09 18:46:01.567543 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.567507 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:01.626132 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.626068 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt"] Jul 09 18:46:01.658318 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.658284 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt"] Jul 09 18:46:01.658454 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.658391 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" Jul 09 18:46:01.661271 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.661248 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 18:46:01.661398 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.661364 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 18:46:01.661977 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.661933 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9g72z\"" Jul 09 18:46:01.668012 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.667992 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:01.762417 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.762390 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5k8f\" (UniqueName: \"kubernetes.io/projected/d551c345-9a33-408c-b17e-e394101f94be-kube-api-access-g5k8f\") pod \"migrator-7f59b54f6c-zptbt\" (UID: \"d551c345-9a33-408c-b17e-e394101f94be\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" Jul 09 18:46:01.768178 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.768154 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:01.784890 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.784870 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/1.log" Jul 09 18:46:01.785189 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.785176 2587 scope.go:117] "RemoveContainer" containerID="15fd6348756ee0684b33d1b5f5a797344123bf0955411e963d46b358de6c9883" Jul 09 18:46:01.785359 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:01.785340 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-8g5bp_openshift-console-operator(de304309-203f-46ba-ba4d-024e0ab59b8c)\"" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podUID="de304309-203f-46ba-ba4d-024e0ab59b8c" Jul 09 18:46:01.863610 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.863581 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5k8f\" (UniqueName: \"kubernetes.io/projected/d551c345-9a33-408c-b17e-e394101f94be-kube-api-access-g5k8f\") pod \"migrator-7f59b54f6c-zptbt\" (UID: \"d551c345-9a33-408c-b17e-e394101f94be\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" Jul 09 18:46:01.873120 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.873098 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5k8f\" (UniqueName: \"kubernetes.io/projected/d551c345-9a33-408c-b17e-e394101f94be-kube-api-access-g5k8f\") pod \"migrator-7f59b54f6c-zptbt\" (UID: \"d551c345-9a33-408c-b17e-e394101f94be\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" Jul 09 18:46:01.967792 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:01.967769 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" Jul 09 18:46:02.030606 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.030580 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:02.111008 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.110979 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt"] Jul 09 18:46:02.113972 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:02.113931 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd551c345_9a33_408c_b17e_e394101f94be.slice/crio-627c5b2106723cd493bd1fad043ddf53edbd3c8054183e07a45f01a6a09663ad WatchSource:0}: Error finding container 627c5b2106723cd493bd1fad043ddf53edbd3c8054183e07a45f01a6a09663ad: Status 404 returned error can't find the container with id 627c5b2106723cd493bd1fad043ddf53edbd3c8054183e07a45f01a6a09663ad Jul 09 18:46:02.118829 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.118810 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:02.469626 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.469591 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:02.469794 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.469686 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:02.469794 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:02.469739 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:46:02.469794 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:02.469751 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:46:02.469977 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:02.469803 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:04.469787665 +0000 UTC m=+50.970746681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : secret "insights-runtime-extractor-tls" not found Jul 09 18:46:02.469977 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:02.469821 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:04.469813142 +0000 UTC m=+50.970772158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : configmap "kube-rbac-proxy" not found Jul 09 18:46:02.571052 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.571025 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r2x85_7fb5d87d-2207-4215-900e-9f5cbde935de/dns-node-resolver/0.log" Jul 09 18:46:02.790181 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.790089 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" event={"ID":"d551c345-9a33-408c-b17e-e394101f94be","Type":"ContainerStarted","Data":"627c5b2106723cd493bd1fad043ddf53edbd3c8054183e07a45f01a6a09663ad"} Jul 09 18:46:02.968753 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:02.968720 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dh7hx_542ce291-5b21-4361-b4a1-58e67e29c530/node-ca/0.log" Jul 09 18:46:03.159555 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.159518 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:03.220005 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.219976 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz"] Jul 09 18:46:03.254865 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.254835 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz"] Jul 09 18:46:03.255049 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.254937 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" Jul 09 18:46:03.258982 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.258937 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5f4jw\"" Jul 09 18:46:03.377674 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.377645 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvxxb\" (UniqueName: \"kubernetes.io/projected/2c1db536-882d-43c5-b1a4-c0f8bb2557e2-kube-api-access-wvxxb\") pod \"network-check-source-869dcfcdcd-d9ldz\" (UID: \"2c1db536-882d-43c5-b1a4-c0f8bb2557e2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" Jul 09 18:46:03.478108 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.478038 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvxxb\" (UniqueName: \"kubernetes.io/projected/2c1db536-882d-43c5-b1a4-c0f8bb2557e2-kube-api-access-wvxxb\") pod \"network-check-source-869dcfcdcd-d9ldz\" (UID: \"2c1db536-882d-43c5-b1a4-c0f8bb2557e2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" Jul 09 18:46:03.488213 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.488191 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvxxb\" (UniqueName: \"kubernetes.io/projected/2c1db536-882d-43c5-b1a4-c0f8bb2557e2-kube-api-access-wvxxb\") pod \"network-check-source-869dcfcdcd-d9ldz\" (UID: \"2c1db536-882d-43c5-b1a4-c0f8bb2557e2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" Jul 09 18:46:03.557536 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.557507 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:46:03.557653 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.557559 2587 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:46:03.557866 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.557855 2587 scope.go:117] "RemoveContainer" containerID="15fd6348756ee0684b33d1b5f5a797344123bf0955411e963d46b358de6c9883" Jul 09 18:46:03.558040 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:03.558022 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-8g5bp_openshift-console-operator(de304309-203f-46ba-ba4d-024e0ab59b8c)\"" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podUID="de304309-203f-46ba-ba4d-024e0ab59b8c" Jul 09 18:46:03.566005 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.565984 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" Jul 09 18:46:03.733631 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.733552 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:03.834388 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.834361 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:03.892968 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.892871 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz"] Jul 09 18:46:03.895310 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:03.895286 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1db536_882d_43c5_b1a4_c0f8bb2557e2.slice/crio-c23c30fb5effccd3867efa7cecfc9d42013ed2fe75aff9002933e343643a6c6d WatchSource:0}: Error finding container c23c30fb5effccd3867efa7cecfc9d42013ed2fe75aff9002933e343643a6c6d: Status 404 returned error can't find the container with id c23c30fb5effccd3867efa7cecfc9d42013ed2fe75aff9002933e343643a6c6d Jul 09 18:46:03.968257 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:03.968236 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wlxp4_6fcf474d-2db5-4a19-803c-414e4ab64a2b/serve-healthcheck-canary/0.log" Jul 09 18:46:04.285233 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.285150 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:04.485938 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.485907 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:04.486116 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.485974 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:04.486116 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:04.486033 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:46:04.486223 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:04.486118 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:08.486098642 +0000 UTC m=+54.987057679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : configmap "kube-rbac-proxy" not found Jul 09 18:46:04.486223 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:04.486169 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:46:04.486316 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:04.486225 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:08.486212212 +0000 UTC m=+54.987171243 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : secret "insights-runtime-extractor-tls" not found Jul 09 18:46:04.662995 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.662965 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:04.799510 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.799436 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" event={"ID":"d551c345-9a33-408c-b17e-e394101f94be","Type":"ContainerStarted","Data":"1e5d64080c017bad99c4f4c281f31747cd448222cc0cd2fa5dc735ba83e09c44"} Jul 09 18:46:04.799988 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.799538 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" event={"ID":"d551c345-9a33-408c-b17e-e394101f94be","Type":"ContainerStarted","Data":"290da55c71172332a892678947bf8fffc8d8bbbcdd5ac325479f3c6b81aba63a"} Jul 09 18:46:04.800787 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.800758 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" event={"ID":"2c1db536-882d-43c5-b1a4-c0f8bb2557e2","Type":"ContainerStarted","Data":"c23c30fb5effccd3867efa7cecfc9d42013ed2fe75aff9002933e343643a6c6d"} Jul 09 18:46:04.816858 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:04.816805 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-zptbt" podStartSLOduration=2.155615871 podStartE2EDuration="3.816785382s" totalImagesPullingTime="1.661169511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:04.816246528 +0000 UTC m=+51.317205590" watchObservedRunningTime="2026-07-09 18:46:04.816785382 +0000 UTC m=+51.317744423" Jul 09 18:46:07.107216 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:07.107185 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:46:07.107658 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:07.107367 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 18:46:07.107658 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:07.107455 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls podName:949afb45-35a9-4750-99a8-9bf32d8f8e25 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:23.107432922 +0000 UTC m=+69.608391956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls") pod "dns-default-9p7tm" (UID: "949afb45-35a9-4750-99a8-9bf32d8f8e25") : secret "dns-default-metrics-tls" not found Jul 09 18:46:07.678908 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:07.678870 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:07.811764 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:07.811725 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" event={"ID":"2c1db536-882d-43c5-b1a4-c0f8bb2557e2","Type":"ContainerStarted","Data":"6426ed3557ae51d2043c681aa0159f529b9eb0bf971afbffa6d0f48a28b16a3c"} Jul 09 18:46:07.828012 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:07.827970 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-d9ldz" podStartSLOduration=1.5950386 podStartE2EDuration="4.827938914s" totalImagesPullingTime="3.232900314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:07.827273518 +0000 UTC m=+54.328232555" watchObservedRunningTime="2026-07-09 18:46:07.827938914 +0000 UTC m=+54.328897951" Jul 09 18:46:08.292332 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.292302 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:08.430848 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.430803 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:08.520041 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.520009 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:08.520258 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.520052 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:08.520258 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:08.520141 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 18:46:08.520258 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:08.520204 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:46:08.520258 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:08.520206 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:16.520188332 +0000 UTC m=+63.021147347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : configmap "kube-rbac-proxy" not found Jul 09 18:46:08.520258 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:08.520260 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:16.520244528 +0000 UTC m=+63.021203546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : secret "insights-runtime-extractor-tls" not found Jul 09 18:46:08.530291 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.530253 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:08.632101 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.632014 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:08.776337 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.776285 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:08.862961 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.862908 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:08.988069 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:08.988038 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:09.126368 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:09.126336 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:46:09.128630 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:09.128597 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd5de19d-1436-4636-9660-008223b3a68e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kwxmz\" (UID: \"cd5de19d-1436-4636-9660-008223b3a68e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:46:09.234350 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:09.234323 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" Jul 09 18:46:09.376435 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:09.376412 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz"] Jul 09 18:46:09.779456 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:09.779422 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:09.818890 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:09.818855 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" event={"ID":"cd5de19d-1436-4636-9660-008223b3a68e","Type":"ContainerStarted","Data":"414a18784ab69412227a76a4013d534148f728c7644f9f6ba1761c6cf55ae4ad"} Jul 09 18:46:10.397179 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:10.397143 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:10.680649 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:10.679697 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:10.865352 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:10.865319 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:10.992416 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:10.992331 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:11.178435 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:11.178405 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:11.789089 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:11.789060 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:11.827197 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:11.827167 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" event={"ID":"cd5de19d-1436-4636-9660-008223b3a68e","Type":"ContainerStarted","Data":"d58f414ad5959defe8792f41a31c43df88b4eaaaf34554c0ae493800606ab7f5"} Jul 09 18:46:11.827342 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:11.827228 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" event={"ID":"cd5de19d-1436-4636-9660-008223b3a68e","Type":"ContainerStarted","Data":"456ce3936d143114158b0e450979d7430ffb853f10dbb22b93c7d1b113fb0f60"} Jul 09 18:46:11.845384 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:11.845341 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kwxmz" podStartSLOduration=17.030761845 podStartE2EDuration="18.84532709s" totalImagesPullingTime="1.814565245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:11.844177329 +0000 UTC m=+58.345136385" watchObservedRunningTime="2026-07-09 18:46:11.84532709 +0000 UTC m=+58.346286127" Jul 09 18:46:11.982956 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:11.982917 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:12.183815 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:12.183785 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:12.384976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:12.384920 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:13.031902 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:13.031867 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:13.130125 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:13.130099 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:13.386894 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:13.386818 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:13.787343 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:13.787313 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:13.985927 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:13.985892 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:14.087784 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:14.087719 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:14.380917 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:14.380840 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:14.535069 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:14.535040 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:14.982055 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:14.982022 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:15.185017 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:15.184982 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:15.389391 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:15.389311 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:16.207917 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:16.207884 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:16.384222 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:16.384193 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:16.592100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:16.592009 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:16.592100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:16.592055 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:16.592286 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:16.592155 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 18:46:16.592286 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:16.592208 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls podName:026a8fa5-9e41-4153-98f4-439bb0f833a9 nodeName:}" failed. No retries permitted until 2026-07-09 18:46:32.592194009 +0000 UTC m=+79.093153025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6wkrq" (UID: "026a8fa5-9e41-4153-98f4-439bb0f833a9") : secret "insights-runtime-extractor-tls" not found Jul 09 18:46:16.592600 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:16.592582 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/026a8fa5-9e41-4153-98f4-439bb0f833a9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:16.723415 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:16.723392 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qpr66" Jul 09 18:46:16.786000 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:16.785967 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:17.060413 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.060384 2587 scope.go:117] "RemoveContainer" containerID="15fd6348756ee0684b33d1b5f5a797344123bf0955411e963d46b358de6c9883" Jul 09 18:46:17.577888 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.577855 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:17.783242 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.783211 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:17.846682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.846626 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 18:46:17.847057 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.847043 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/1.log" Jul 09 18:46:17.847113 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.847074 2587 generic.go:356] "Generic (PLEG): container finished" podID="de304309-203f-46ba-ba4d-024e0ab59b8c" containerID="148459c9070631f483a74155462d393c48beb68b8720c1785b0e7543b298f98b" exitCode=255 Jul 09 18:46:17.847113 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.847107 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8g5bp" event={"ID":"de304309-203f-46ba-ba4d-024e0ab59b8c","Type":"ContainerDied","Data":"148459c9070631f483a74155462d393c48beb68b8720c1785b0e7543b298f98b"} Jul 09 18:46:17.847170 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.847152 2587 scope.go:117] "RemoveContainer" containerID="15fd6348756ee0684b33d1b5f5a797344123bf0955411e963d46b358de6c9883" Jul 09 18:46:17.847471 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:17.847442 2587 scope.go:117] "RemoveContainer" containerID="148459c9070631f483a74155462d393c48beb68b8720c1785b0e7543b298f98b" Jul 09 18:46:17.847619 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:17.847597 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-8g5bp_openshift-console-operator(de304309-203f-46ba-ba4d-024e0ab59b8c)\"" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podUID="de304309-203f-46ba-ba4d-024e0ab59b8c" Jul 09 18:46:18.183198 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:18.183169 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:18.389716 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:18.389659 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:18.779123 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:18.779088 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:18.852134 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:18.852109 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 18:46:18.986458 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:18.986415 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:20.420186 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.420154 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:46:20.420615 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.420199 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:46:20.423394 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.423379 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:46:20.424449 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.424432 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:46:20.433012 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.432988 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f30cc372-130d-417c-a1c4-2754e9d9f80d-metrics-certs\") pod \"network-metrics-daemon-b62dq\" (UID: \"f30cc372-130d-417c-a1c4-2754e9d9f80d\") " pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:46:20.433098 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.432990 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88545bfb-fc18-4f99-ab53-894af89974a2-original-pull-secret\") pod \"global-pull-secret-syncer-vf5zs\" (UID: \"88545bfb-fc18-4f99-ab53-894af89974a2\") " pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:46:20.521503 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.521460 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:46:20.523883 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.523856 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcfpz\" (UniqueName: \"kubernetes.io/projected/fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4-kube-api-access-mcfpz\") pod \"network-check-target-l5vkd\" (UID: \"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4\") " pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:46:20.675675 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.675598 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-prgmc\"" Jul 09 18:46:20.679759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.679741 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vf5zs" Jul 09 18:46:20.684334 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.684314 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:46:20.688609 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.688591 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ckpt2\"" Jul 09 18:46:20.696372 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.696353 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62dq" Jul 09 18:46:20.823023 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.819926 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vf5zs"] Jul 09 18:46:20.826796 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:20.826771 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88545bfb_fc18_4f99_ab53_894af89974a2.slice/crio-8a1b88d68a920ba658ea5f96097aa7511faea2ae9a5991ebf3b978925f578cc2 WatchSource:0}: Error finding container 8a1b88d68a920ba658ea5f96097aa7511faea2ae9a5991ebf3b978925f578cc2: Status 404 returned error can't find the container with id 8a1b88d68a920ba658ea5f96097aa7511faea2ae9a5991ebf3b978925f578cc2 Jul 09 18:46:20.833810 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.833789 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5vkd"] Jul 09 18:46:20.836602 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:20.836569 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe86fd8_0a52_4b28_b3b0_bbd648ae91a4.slice/crio-357af6b4e3ab31ba9d80887124ba7d0f7c608e5816f1f753c54efc1767eb42a2 WatchSource:0}: Error finding container 357af6b4e3ab31ba9d80887124ba7d0f7c608e5816f1f753c54efc1767eb42a2: Status 404 returned error can't find the container with id 357af6b4e3ab31ba9d80887124ba7d0f7c608e5816f1f753c54efc1767eb42a2 Jul 09 18:46:20.853389 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.853366 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b62dq"] Jul 09 18:46:20.860318 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.860287 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vf5zs" event={"ID":"88545bfb-fc18-4f99-ab53-894af89974a2","Type":"ContainerStarted","Data":"8a1b88d68a920ba658ea5f96097aa7511faea2ae9a5991ebf3b978925f578cc2"} Jul 09 18:46:20.861324 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:20.861303 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5vkd" event={"ID":"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4","Type":"ContainerStarted","Data":"357af6b4e3ab31ba9d80887124ba7d0f7c608e5816f1f753c54efc1767eb42a2"} Jul 09 18:46:20.863437 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:20.863414 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30cc372_130d_417c_a1c4_2754e9d9f80d.slice/crio-84d63466d5691b24449753777400060e5de5703af84d9c72e4d1889f6cb40f8c WatchSource:0}: Error finding container 84d63466d5691b24449753777400060e5de5703af84d9c72e4d1889f6cb40f8c: Status 404 returned error can't find the container with id 84d63466d5691b24449753777400060e5de5703af84d9c72e4d1889f6cb40f8c Jul 09 18:46:21.867266 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:21.867186 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b62dq" event={"ID":"f30cc372-130d-417c-a1c4-2754e9d9f80d","Type":"ContainerStarted","Data":"84d63466d5691b24449753777400060e5de5703af84d9c72e4d1889f6cb40f8c"} Jul 09 18:46:21.869011 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:21.868979 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5vkd" event={"ID":"fbe86fd8-0a52-4b28-b3b0-bbd648ae91a4","Type":"ContainerStarted","Data":"19d89262220c4a1c5f5a09700ac969fba6a58e271075e85ac5497fca8a9068d0"} Jul 09 18:46:21.869155 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:21.869088 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:46:21.890320 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:21.890263 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l5vkd" podStartSLOduration=67.890242804 podStartE2EDuration="1m7.890242804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:21.889357314 +0000 UTC m=+68.390316382" watchObservedRunningTime="2026-07-09 18:46:21.890242804 +0000 UTC m=+68.391201845" Jul 09 18:46:22.874997 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:22.874960 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b62dq" event={"ID":"f30cc372-130d-417c-a1c4-2754e9d9f80d","Type":"ContainerStarted","Data":"27484c65a9720bbae299d464a7be9149c9ef6888f8fd8fa6f860d1d76e0bd0c1"} Jul 09 18:46:22.875394 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:22.875024 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b62dq" event={"ID":"f30cc372-130d-417c-a1c4-2754e9d9f80d","Type":"ContainerStarted","Data":"ec8907fcb025991a04b9b72d75ab1b23517978b81acd7a20d159adad144a884e"} Jul 09 18:46:22.894317 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:22.894271 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b62dq" podStartSLOduration=66.814621707 podStartE2EDuration="1m7.894251031s" totalImagesPullingTime="1.079629324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:22.893162226 +0000 UTC m=+69.394121285" watchObservedRunningTime="2026-07-09 18:46:22.894251031 +0000 UTC m=+69.395210070" Jul 09 18:46:23.143432 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.143355 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:46:23.145567 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.145549 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/949afb45-35a9-4750-99a8-9bf32d8f8e25-metrics-tls\") pod \"dns-default-9p7tm\" (UID: \"949afb45-35a9-4750-99a8-9bf32d8f8e25\") " pod="openshift-dns/dns-default-9p7tm" Jul 09 18:46:23.410524 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.410492 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5t98t\"" Jul 09 18:46:23.417953 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.417919 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9p7tm" Jul 09 18:46:23.557746 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.557703 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:46:23.557900 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.557792 2587 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:46:23.558127 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.558111 2587 scope.go:117] "RemoveContainer" containerID="148459c9070631f483a74155462d393c48beb68b8720c1785b0e7543b298f98b" Jul 09 18:46:23.558321 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:23.558296 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-8g5bp_openshift-console-operator(de304309-203f-46ba-ba4d-024e0ab59b8c)\"" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podUID="de304309-203f-46ba-ba4d-024e0ab59b8c" Jul 09 18:46:23.566912 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.566891 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9p7tm"] Jul 09 18:46:23.569684 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:23.569659 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949afb45_35a9_4750_99a8_9bf32d8f8e25.slice/crio-b72ae1a311ff144d0b659b6b4ba02dc3199ee8edd801227891a1b6e44a466e66 WatchSource:0}: Error finding container b72ae1a311ff144d0b659b6b4ba02dc3199ee8edd801227891a1b6e44a466e66: Status 404 returned error can't find the container with id b72ae1a311ff144d0b659b6b4ba02dc3199ee8edd801227891a1b6e44a466e66 Jul 09 18:46:23.879305 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.879221 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p7tm" event={"ID":"949afb45-35a9-4750-99a8-9bf32d8f8e25","Type":"ContainerStarted","Data":"b72ae1a311ff144d0b659b6b4ba02dc3199ee8edd801227891a1b6e44a466e66"} Jul 09 18:46:23.879708 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:23.879471 2587 scope.go:117] "RemoveContainer" containerID="148459c9070631f483a74155462d393c48beb68b8720c1785b0e7543b298f98b" Jul 09 18:46:23.879708 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:23.879610 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-8g5bp_openshift-console-operator(de304309-203f-46ba-ba4d-024e0ab59b8c)\"" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podUID="de304309-203f-46ba-ba4d-024e0ab59b8c" Jul 09 18:46:24.759705 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.759667 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb"] Jul 09 18:46:24.782092 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.782063 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g"] Jul 09 18:46:24.782261 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.782201 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:24.785012 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.784986 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 18:46:24.785466 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.785377 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 18:46:24.785608 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.785586 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5wzqs\"" Jul 09 18:46:24.785681 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.785608 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 18:46:24.786999 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.786897 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 18:46:24.796197 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.796177 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb"] Jul 09 18:46:24.796290 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.796202 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g"] Jul 09 18:46:24.796290 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.796278 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.799064 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.799017 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 18:46:24.799302 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.799275 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 18:46:24.799406 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.799359 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 18:46:24.799590 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.799573 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 18:46:24.860010 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.859786 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-ca\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.860010 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.859924 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b5e7572-95e5-4d00-9ccc-d69467e4b4f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-679845897b-lfnqb\" (UID: \"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:24.860010 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.860002 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-hub\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.860301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.860026 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dkx9\" (UniqueName: \"kubernetes.io/projected/c5fdc3aa-0816-42f7-8610-0f3d15177251-kube-api-access-8dkx9\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.860301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.860049 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.860301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.860087 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5fdc3aa-0816-42f7-8610-0f3d15177251-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.860301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.860112 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.860301 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.860157 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7bh5\" (UniqueName: \"kubernetes.io/projected/6b5e7572-95e5-4d00-9ccc-d69467e4b4f4-kube-api-access-v7bh5\") pod \"managed-serviceaccount-addon-agent-679845897b-lfnqb\" (UID: \"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:24.861645 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.861622 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6745f944d7-jmsjg"] Jul 09 18:46:24.880414 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.880387 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.881189 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.881165 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6745f944d7-jmsjg"] Jul 09 18:46:24.884434 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.884409 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:24.884728 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.884705 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 18:46:24.884826 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.884748 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 18:46:24.885143 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.885126 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l92kj\"" Jul 09 18:46:24.885256 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.885226 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 18:46:24.889864 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.889844 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 18:46:24.960867 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.960836 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-ca\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.961052 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.960875 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c8627e74-def1-4cd6-8561-440a4f05b355-registry-certificates\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.961052 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.960904 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b5e7572-95e5-4d00-9ccc-d69467e4b4f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-679845897b-lfnqb\" (UID: \"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:24.961052 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.960926 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c8627e74-def1-4cd6-8561-440a4f05b355-image-registry-private-configuration\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.961213 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961057 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-hub\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.961213 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961093 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dkx9\" (UniqueName: \"kubernetes.io/projected/c5fdc3aa-0816-42f7-8610-0f3d15177251-kube-api-access-8dkx9\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.961213 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961120 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.961213 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961158 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-registry-tls\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.961213 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961188 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-bound-sa-token\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.961514 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961218 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5fdc3aa-0816-42f7-8610-0f3d15177251-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.961514 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961245 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.961514 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961287 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c8627e74-def1-4cd6-8561-440a4f05b355-ca-trust-extracted\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.961514 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961311 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8627e74-def1-4cd6-8561-440a4f05b355-trusted-ca\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.961514 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961346 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7bh5\" (UniqueName: \"kubernetes.io/projected/6b5e7572-95e5-4d00-9ccc-d69467e4b4f4-kube-api-access-v7bh5\") pod \"managed-serviceaccount-addon-agent-679845897b-lfnqb\" (UID: \"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:24.961514 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961372 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c8627e74-def1-4cd6-8561-440a4f05b355-installation-pull-secrets\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.961514 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.961401 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt5xd\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-kube-api-access-wt5xd\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:24.962205 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.962164 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5fdc3aa-0816-42f7-8610-0f3d15177251-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.963996 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.963974 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-ca\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.964225 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.964203 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-hub\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.964326 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.964235 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b5e7572-95e5-4d00-9ccc-d69467e4b4f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-679845897b-lfnqb\" (UID: \"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:24.964455 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.964430 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.964626 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.964604 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5fdc3aa-0816-42f7-8610-0f3d15177251-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.974064 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.974031 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dkx9\" (UniqueName: \"kubernetes.io/projected/c5fdc3aa-0816-42f7-8610-0f3d15177251-kube-api-access-8dkx9\") pod \"cluster-proxy-proxy-agent-6d6dd45c7-dpg7g\" (UID: \"c5fdc3aa-0816-42f7-8610-0f3d15177251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:24.974647 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.974618 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7bh5\" (UniqueName: \"kubernetes.io/projected/6b5e7572-95e5-4d00-9ccc-d69467e4b4f4-kube-api-access-v7bh5\") pod \"managed-serviceaccount-addon-agent-679845897b-lfnqb\" (UID: \"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:24.975410 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:24.975384 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:25.062253 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062170 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c8627e74-def1-4cd6-8561-440a4f05b355-installation-pull-secrets\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.062253 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062204 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt5xd\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-kube-api-access-wt5xd\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.062461 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062260 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c8627e74-def1-4cd6-8561-440a4f05b355-registry-certificates\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.062461 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062279 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c8627e74-def1-4cd6-8561-440a4f05b355-image-registry-private-configuration\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.062461 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062328 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-registry-tls\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.062461 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062349 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-bound-sa-token\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.062461 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062388 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c8627e74-def1-4cd6-8561-440a4f05b355-ca-trust-extracted\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.062461 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062412 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8627e74-def1-4cd6-8561-440a4f05b355-trusted-ca\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.063042 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.062996 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c8627e74-def1-4cd6-8561-440a4f05b355-ca-trust-extracted\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.063462 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.063411 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c8627e74-def1-4cd6-8561-440a4f05b355-registry-certificates\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.063700 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.063681 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8627e74-def1-4cd6-8561-440a4f05b355-trusted-ca\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.065159 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.065129 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c8627e74-def1-4cd6-8561-440a4f05b355-installation-pull-secrets\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.065307 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.065289 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c8627e74-def1-4cd6-8561-440a4f05b355-image-registry-private-configuration\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.065352 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.065333 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-registry-tls\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.071178 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.071154 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:25.073475 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.073454 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt5xd\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-kube-api-access-wt5xd\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.073669 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.073645 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8627e74-def1-4cd6-8561-440a4f05b355-bound-sa-token\") pod \"image-registry-6745f944d7-jmsjg\" (UID: \"c8627e74-def1-4cd6-8561-440a4f05b355\") " pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.101093 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.101071 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" Jul 09 18:46:25.109271 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.109231 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" Jul 09 18:46:25.171420 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.171387 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:25.200349 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.200320 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.271698 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.271666 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:25.335389 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.335350 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:25.471384 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.471356 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb"] Jul 09 18:46:25.627114 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:25.627036 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b5e7572_95e5_4d00_9ccc_d69467e4b4f4.slice/crio-48834945998ae0a3b5700fa7d6afa21683a5355442872fb9a69145c8a6794433 WatchSource:0}: Error finding container 48834945998ae0a3b5700fa7d6afa21683a5355442872fb9a69145c8a6794433: Status 404 returned error can't find the container with id 48834945998ae0a3b5700fa7d6afa21683a5355442872fb9a69145c8a6794433 Jul 09 18:46:25.702522 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.701619 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6745f944d7-jmsjg"] Jul 09 18:46:25.702522 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.701717 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g"] Jul 09 18:46:25.703568 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:25.703538 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8627e74_def1_4cd6_8561_440a4f05b355.slice/crio-44f489dc9cab512e3a4e4076c881b9a1bf81dfe7c0a0de9b7b109ef06e044c38 WatchSource:0}: Error finding container 44f489dc9cab512e3a4e4076c881b9a1bf81dfe7c0a0de9b7b109ef06e044c38: Status 404 returned error can't find the container with id 44f489dc9cab512e3a4e4076c881b9a1bf81dfe7c0a0de9b7b109ef06e044c38 Jul 09 18:46:25.715887 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:25.715806 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5fdc3aa_0816_42f7_8610_0f3d15177251.slice/crio-af6d308317232fd1ebaf563251c7908cb85412d530d72764517fc35d9ff2e9c1 WatchSource:0}: Error finding container af6d308317232fd1ebaf563251c7908cb85412d530d72764517fc35d9ff2e9c1: Status 404 returned error can't find the container with id af6d308317232fd1ebaf563251c7908cb85412d530d72764517fc35d9ff2e9c1 Jul 09 18:46:25.889008 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.888909 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" event={"ID":"c5fdc3aa-0816-42f7-8610-0f3d15177251","Type":"ContainerStarted","Data":"af6d308317232fd1ebaf563251c7908cb85412d530d72764517fc35d9ff2e9c1"} Jul 09 18:46:25.890776 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.890740 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vf5zs" event={"ID":"88545bfb-fc18-4f99-ab53-894af89974a2","Type":"ContainerStarted","Data":"19f8fb094069ab8f510850dcb055d8fa235ba5529e54287520ebc25a4e18bf3a"} Jul 09 18:46:25.892590 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.892510 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" event={"ID":"c8627e74-def1-4cd6-8561-440a4f05b355","Type":"ContainerStarted","Data":"f5453966b846b329e46b621a50b09f4e167d60d7651b61c3f040b2831f8a4ddd"} Jul 09 18:46:25.892590 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.892554 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" event={"ID":"c8627e74-def1-4cd6-8561-440a4f05b355","Type":"ContainerStarted","Data":"44f489dc9cab512e3a4e4076c881b9a1bf81dfe7c0a0de9b7b109ef06e044c38"} Jul 09 18:46:25.892590 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.892580 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:25.893864 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.893846 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" event={"ID":"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4","Type":"ContainerStarted","Data":"48834945998ae0a3b5700fa7d6afa21683a5355442872fb9a69145c8a6794433"} Jul 09 18:46:25.909453 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.909411 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vf5zs" podStartSLOduration=66.075433458 podStartE2EDuration="1m10.909398207s" totalImagesPullingTime="4.833964749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:25.90906743 +0000 UTC m=+72.410026490" watchObservedRunningTime="2026-07-09 18:46:25.909398207 +0000 UTC m=+72.410357236" Jul 09 18:46:25.930801 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:25.930748 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" podStartSLOduration=1.9307299740000001 podStartE2EDuration="1.930729974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:25.929830351 +0000 UTC m=+72.430789428" watchObservedRunningTime="2026-07-09 18:46:25.930729974 +0000 UTC m=+72.431689008" Jul 09 18:46:27.439109 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:27.439073 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:27.553518 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:27.553483 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:29.897170 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:29.897134 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:29.913399 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:29.913363 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p7tm" event={"ID":"949afb45-35a9-4750-99a8-9bf32d8f8e25","Type":"ContainerStarted","Data":"acffba399f81f31eddaef0e149dc023d7207a7ec42abf3902fbca486ea8e7861"} Jul 09 18:46:29.913567 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:29.913469 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p7tm" event={"ID":"949afb45-35a9-4750-99a8-9bf32d8f8e25","Type":"ContainerStarted","Data":"a7b9b41ec513e177374db715297c0856c4ad105d8a96dc010dfded077f83a3d4"} Jul 09 18:46:29.913567 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:29.913501 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9p7tm" Jul 09 18:46:29.940296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:29.940238 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9p7tm" podStartSLOduration=33.714909157 podStartE2EDuration="38.940219637s" totalImagesPullingTime="5.22531048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:29.938700646 +0000 UTC m=+76.439659699" watchObservedRunningTime="2026-07-09 18:46:29.940219637 +0000 UTC m=+76.441178670" Jul 09 18:46:30.000747 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:30.000716 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:30.918118 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:30.918091 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" event={"ID":"c5fdc3aa-0816-42f7-8610-0f3d15177251","Type":"ContainerStarted","Data":"0d32fdc8cf83d448c6226d91d106503a0fca12c12071bc817677f1ac9a506cb6"} Jul 09 18:46:30.919606 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:30.919575 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" event={"ID":"6b5e7572-95e5-4d00-9ccc-d69467e4b4f4","Type":"ContainerStarted","Data":"8648fb98f119ef4e2e128d443e5456c8fd74c967d046d1f0b94bd27e76044bbc"} Jul 09 18:46:30.942786 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:30.942738 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-679845897b-lfnqb" podStartSLOduration=3.429616788 podStartE2EDuration="6.942724316s" totalImagesPullingTime="3.513107528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:30.941606756 +0000 UTC m=+77.442565794" watchObservedRunningTime="2026-07-09 18:46:30.942724316 +0000 UTC m=+77.443683353" Jul 09 18:46:31.507385 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:31.507345 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:31.707826 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:31.707790 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:31.823519 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:31.823432 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:31.925607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:31.925577 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" event={"ID":"c5fdc3aa-0816-42f7-8610-0f3d15177251","Type":"ContainerStarted","Data":"d70be57f3aa26e22b20a50593f2e8115cb5493cfc2acfbdb9068b3fa4d0d9b33"} Jul 09 18:46:31.925607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:31.925620 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" event={"ID":"c5fdc3aa-0816-42f7-8610-0f3d15177251","Type":"ContainerStarted","Data":"385846a480c0ab7360098cf06bff87fc033980af170e09a0b871e8b05a55f499"} Jul 09 18:46:31.948020 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:31.947978 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d6dd45c7-dpg7g" podStartSLOduration=2.863720693 podStartE2EDuration="7.947964714s" totalImagesPullingTime="5.084244021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:31.946504282 +0000 UTC m=+78.447463320" watchObservedRunningTime="2026-07-09 18:46:31.947964714 +0000 UTC m=+78.448923748" Jul 09 18:46:32.436788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:32.436755 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:32.502277 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:32.502244 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:32.625480 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:32.625446 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:32.627660 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:32.627633 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/026a8fa5-9e41-4153-98f4-439bb0f833a9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6wkrq\" (UID: \"026a8fa5-9e41-4153-98f4-439bb0f833a9\") " pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:32.918693 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:32.918666 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qssxl\"" Jul 09 18:46:32.924312 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:32.924298 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6wkrq" Jul 09 18:46:33.058975 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:33.058931 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6wkrq"] Jul 09 18:46:33.061500 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:33.061475 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026a8fa5_9e41_4153_98f4_439bb0f833a9.slice/crio-82b1c5815e7a8ecd8e5449152850e2e9457386d2b597a6f956d4bb23b2b39a72 WatchSource:0}: Error finding container 82b1c5815e7a8ecd8e5449152850e2e9457386d2b597a6f956d4bb23b2b39a72: Status 404 returned error can't find the container with id 82b1c5815e7a8ecd8e5449152850e2e9457386d2b597a6f956d4bb23b2b39a72 Jul 09 18:46:33.268667 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:33.268572 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:33.335142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:33.335113 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:33.940004 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:33.939896 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6wkrq" event={"ID":"026a8fa5-9e41-4153-98f4-439bb0f833a9","Type":"ContainerStarted","Data":"82b1c5815e7a8ecd8e5449152850e2e9457386d2b597a6f956d4bb23b2b39a72"} Jul 09 18:46:35.060544 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:35.060514 2587 scope.go:117] "RemoveContainer" containerID="148459c9070631f483a74155462d393c48beb68b8720c1785b0e7543b298f98b" Jul 09 18:46:35.060919 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:46:35.060697 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-8g5bp_openshift-console-operator(de304309-203f-46ba-ba4d-024e0ab59b8c)\"" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podUID="de304309-203f-46ba-ba4d-024e0ab59b8c" Jul 09 18:46:36.523216 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:36.523078 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:37.957486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:37.957453 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6wkrq" event={"ID":"026a8fa5-9e41-4153-98f4-439bb0f833a9","Type":"ContainerStarted","Data":"1bd2bbbcbc2c4b2ad92c02465323a1b8c4a825655916a36af47a04dcd76ed68e"} Jul 09 18:46:38.753769 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:38.753731 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:38.837393 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:38.837355 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:38.963707 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:38.963671 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6wkrq" event={"ID":"026a8fa5-9e41-4153-98f4-439bb0f833a9","Type":"ContainerStarted","Data":"6504aed6c3dae3cc1105811614be4e872412a4bbc69b99d30b716eaf8b70df70"} Jul 09 18:46:39.127304 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:39.127215 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:39.922540 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:39.922510 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9p7tm" Jul 09 18:46:40.974003 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:40.973964 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6wkrq" event={"ID":"026a8fa5-9e41-4153-98f4-439bb0f833a9","Type":"ContainerStarted","Data":"1ce01db772d7ba7c0ed7794267434511e7b6c01afb186ac26d2e500dbeab0f47"} Jul 09 18:46:41.000558 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:41.000512 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6wkrq" podStartSLOduration=33.739194269 podStartE2EDuration="41.000497976s" totalImagesPullingTime="7.261303707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:46:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:40.999765062 +0000 UTC m=+87.500724101" watchObservedRunningTime="2026-07-09 18:46:41.000497976 +0000 UTC m=+87.501457015" Jul 09 18:46:41.843391 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:41.843361 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:43.948536 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:43.948503 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:44.083653 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.083621 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:44.256584 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.256502 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:44.371455 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.371421 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:44.396919 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.396891 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2sxcb"] Jul 09 18:46:44.402025 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.402005 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.406757 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.406735 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 18:46:44.406757 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.406750 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 18:46:44.407585 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.407564 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 18:46:44.407688 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.407586 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-svpf7\"" Jul 09 18:46:44.408517 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.408403 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 18:46:44.408517 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.408417 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 18:46:44.413175 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.413157 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 18:46:44.513792 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513728 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-textfile\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513792 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513758 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-wtmp\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513792 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513778 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-root\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513969 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513799 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-accelerators-collector-config\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513969 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513869 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa2bf4cf-c6bc-4981-8a80-5f148b082519-metrics-client-ca\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513969 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513890 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-sys\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513969 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513904 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-tls\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513969 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513917 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hldz\" (UniqueName: \"kubernetes.io/projected/fa2bf4cf-c6bc-4981-8a80-5f148b082519-kube-api-access-2hldz\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.513969 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.513937 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.517302 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.517280 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:44.614820 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.614791 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-accelerators-collector-config\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.614978 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.614830 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa2bf4cf-c6bc-4981-8a80-5f148b082519-metrics-client-ca\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.614978 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.614957 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-sys\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615073 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.614989 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-tls\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615073 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615006 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hldz\" (UniqueName: \"kubernetes.io/projected/fa2bf4cf-c6bc-4981-8a80-5f148b082519-kube-api-access-2hldz\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615073 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615035 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615073 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615043 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-sys\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615073 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615073 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-textfile\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615347 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615105 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-wtmp\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615347 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615132 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-root\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615347 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615204 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-root\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615347 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615282 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-wtmp\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615528 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615454 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-textfile\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615528 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615474 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-accelerators-collector-config\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.615528 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.615473 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa2bf4cf-c6bc-4981-8a80-5f148b082519-metrics-client-ca\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.617270 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.617248 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.617354 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.617337 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fa2bf4cf-c6bc-4981-8a80-5f148b082519-node-exporter-tls\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.622031 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.622004 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:44.636688 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.636663 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hldz\" (UniqueName: \"kubernetes.io/projected/fa2bf4cf-c6bc-4981-8a80-5f148b082519-kube-api-access-2hldz\") pod \"node-exporter-2sxcb\" (UID: \"fa2bf4cf-c6bc-4981-8a80-5f148b082519\") " pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.710898 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.710864 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2sxcb" Jul 09 18:46:44.722464 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:46:44.722438 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2bf4cf_c6bc_4981_8a80_5f148b082519.slice/crio-bcf73008eba54603e2d5265a301d8620061bbdeeefb03a0c4635f73af6b3d717 WatchSource:0}: Error finding container bcf73008eba54603e2d5265a301d8620061bbdeeefb03a0c4635f73af6b3d717: Status 404 returned error can't find the container with id bcf73008eba54603e2d5265a301d8620061bbdeeefb03a0c4635f73af6b3d717 Jul 09 18:46:44.830196 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.830115 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:44.987318 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:44.987285 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sxcb" event={"ID":"fa2bf4cf-c6bc-4981-8a80-5f148b082519","Type":"ContainerStarted","Data":"bcf73008eba54603e2d5265a301d8620061bbdeeefb03a0c4635f73af6b3d717"} Jul 09 18:46:45.081404 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.081318 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:45.204537 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.204492 2587 patch_prober.go:28] interesting pod/image-registry-6745f944d7-jmsjg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 18:46:45.204673 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.204543 2587 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" podUID="c8627e74-def1-4cd6-8561-440a4f05b355" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 18:46:45.264939 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.264911 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:45.449623 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.449570 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:45.755833 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.755725 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:45.992164 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.992121 2587 generic.go:356] "Generic (PLEG): container finished" podID="fa2bf4cf-c6bc-4981-8a80-5f148b082519" containerID="47ff9564aa3764d79885387ab92a7c0f3aa9e0f6f5b29e6c5dec449d6083b755" exitCode=0 Jul 09 18:46:45.992483 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:45.992192 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sxcb" event={"ID":"fa2bf4cf-c6bc-4981-8a80-5f148b082519","Type":"ContainerDied","Data":"47ff9564aa3764d79885387ab92a7c0f3aa9e0f6f5b29e6c5dec449d6083b755"} Jul 09 18:46:46.339481 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:46.339390 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:46.902967 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:46.902911 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6745f944d7-jmsjg" Jul 09 18:46:46.998563 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:46.998524 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sxcb" event={"ID":"fa2bf4cf-c6bc-4981-8a80-5f148b082519","Type":"ContainerStarted","Data":"4d8d59f6675bb320e2383dce3dccb617efc6b12435971115bd8ada675a544526"} Jul 09 18:46:46.998930 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:46.998572 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sxcb" event={"ID":"fa2bf4cf-c6bc-4981-8a80-5f148b082519","Type":"ContainerStarted","Data":"916d58d1c2f2c1eb45c94790b342e2a00d487df19b6fbbe3f919df736dbd67e7"} Jul 09 18:46:47.044129 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.044080 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2sxcb" podStartSLOduration=1.873258184 podStartE2EDuration="3.044066529s" totalImagesPullingTime="1.107786768s" totalInitContainerRuntime="63.021577ms" isStatefulPod=true podCreationTimestamp="2026-07-09 18:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:47.043539279 +0000 UTC m=+93.544498318" watchObservedRunningTime="2026-07-09 18:46:47.044066529 +0000 UTC m=+93.545025566" Jul 09 18:46:47.213154 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.213066 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:47.313450 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.313418 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:47.442869 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.442836 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:47.513571 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.513497 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:47.725522 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.725489 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:47.788770 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.788699 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:47.870987 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.870936 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:47.997527 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:47.997497 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:48.060618 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:48.060559 2587 scope.go:117] "RemoveContainer" containerID="148459c9070631f483a74155462d393c48beb68b8720c1785b0e7543b298f98b" Jul 09 18:46:48.640275 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:48.640240 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:48.709029 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:48.708985 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:48.999786 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:48.999751 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:49.007532 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.007507 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 18:46:49.007640 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.007577 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8g5bp" event={"ID":"de304309-203f-46ba-ba4d-024e0ab59b8c","Type":"ContainerStarted","Data":"8846e05df79e329bb84f81287b3f80e43380c2900391d951c70771a84b1cb395"} Jul 09 18:46:49.007908 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.007876 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:46:49.029714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.029691 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-8g5bp" Jul 09 18:46:49.041486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.041438 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-8g5bp" podStartSLOduration=50.291009964 podStartE2EDuration="56.041419944s" totalImagesPullingTime="5.75040998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:46:49.041204181 +0000 UTC m=+95.542163247" watchObservedRunningTime="2026-07-09 18:46:49.041419944 +0000 UTC m=+95.542378984" Jul 09 18:46:49.317193 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.317112 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:49.409603 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.409571 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:49.535230 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.535176 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:49.955261 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:49.955231 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:50.132328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:50.132299 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:50.848492 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:50.848462 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:50.933957 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:50.933924 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:50.997868 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:50.997839 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:51.229350 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:51.229319 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:51.725887 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:51.725858 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:51.874746 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:51.874713 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:51.970092 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:51.970058 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:52.609407 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:52.609373 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:52.687486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:52.687456 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:52.877854 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:52.877772 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l5vkd" Jul 09 18:46:54.626235 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:54.626197 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:54.726763 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:54.726736 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:54.826333 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:54.826305 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:54.923448 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:54.923414 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:55.309573 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:55.309491 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:55.412917 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:55.412887 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:57.477030 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:57.476990 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:57.544581 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:57.544550 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:57.988929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:57.988897 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:58.092551 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:58.092506 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:58.249628 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:58.249557 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:46:58.351559 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:46:58.351524 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:00.156757 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:00.156723 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:00.261322 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:00.261285 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:03.822212 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:03.822180 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:03.924746 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:03.924710 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:04.441725 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:04.441689 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:04.512287 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:04.512252 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:06.068291 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:06.068261 2587 generic.go:356] "Generic (PLEG): container finished" podID="5400ba01-ed2f-40e4-8c04-85be98f44d1a" containerID="5137d4c7949dcd4396ed82f7131f287adc0a9215a07e9f26d713d7f324e315aa" exitCode=0 Jul 09 18:47:06.068685 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:06.068332 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" event={"ID":"5400ba01-ed2f-40e4-8c04-85be98f44d1a","Type":"ContainerDied","Data":"5137d4c7949dcd4396ed82f7131f287adc0a9215a07e9f26d713d7f324e315aa"} Jul 09 18:47:06.068729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:06.068683 2587 scope.go:117] "RemoveContainer" containerID="5137d4c7949dcd4396ed82f7131f287adc0a9215a07e9f26d713d7f324e315aa" Jul 09 18:47:07.074139 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:07.074098 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5r4rb" event={"ID":"5400ba01-ed2f-40e4-8c04-85be98f44d1a","Type":"ContainerStarted","Data":"6eaf9e376139e7cdc792e03b36503a0502f6f396b3a406af2f9367ff49fa67d7"} Jul 09 18:47:08.167236 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:08.167202 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:08.267305 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:08.267275 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:12.096389 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:12.096358 2587 generic.go:356] "Generic (PLEG): container finished" podID="84b890da-cc0c-4274-9b59-7839dad6a083" containerID="0ff1e3d58a578f03d8095778561344e7b9d1d6203dce0b6210f668f4d3e5228f" exitCode=0 Jul 09 18:47:12.096759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:12.096436 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" event={"ID":"84b890da-cc0c-4274-9b59-7839dad6a083","Type":"ContainerDied","Data":"0ff1e3d58a578f03d8095778561344e7b9d1d6203dce0b6210f668f4d3e5228f"} Jul 09 18:47:12.096759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:12.096733 2587 scope.go:117] "RemoveContainer" containerID="0ff1e3d58a578f03d8095778561344e7b9d1d6203dce0b6210f668f4d3e5228f" Jul 09 18:47:12.446724 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:12.446688 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:12.518424 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:12.518392 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:13.101961 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:13.101922 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lj7ph" event={"ID":"84b890da-cc0c-4274-9b59-7839dad6a083","Type":"ContainerStarted","Data":"78fe020199b2c7cf8c1c184fcd413a381da1826967a18e030d8e4d6949814415"} Jul 09 18:47:17.360073 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:17.360042 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:17.459859 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:17.459826 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:26.148501 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:26.148468 2587 generic.go:356] "Generic (PLEG): container finished" podID="6948c276-c69c-4349-bff2-cdec236e5419" containerID="368f048c32c20aac78c808a429f8e644facbd56008b2947355509171352596f1" exitCode=0 Jul 09 18:47:26.148980 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:26.148539 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" event={"ID":"6948c276-c69c-4349-bff2-cdec236e5419","Type":"ContainerDied","Data":"368f048c32c20aac78c808a429f8e644facbd56008b2947355509171352596f1"} Jul 09 18:47:26.148980 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:26.148869 2587 scope.go:117] "RemoveContainer" containerID="368f048c32c20aac78c808a429f8e644facbd56008b2947355509171352596f1" Jul 09 18:47:27.153400 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:27.153369 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-77lvl" event={"ID":"6948c276-c69c-4349-bff2-cdec236e5419","Type":"ContainerStarted","Data":"18fb13941fe8aca1fa6d64115b8a558b2c4185142b8065c0ba6cbdd0d761f78a"} Jul 09 18:47:27.478547 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:27.478474 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:27.583318 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:27.583292 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:40.699083 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:40.699048 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:40.800090 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:40.800051 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:42.447613 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:42.447580 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:47:42.518623 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:47:42.518592 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:03.910375 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:03.910341 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:04.345910 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:04.345834 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:12.759235 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:12.759204 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:12.821233 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:12.821205 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:23.666249 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:23.666215 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:23.766295 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:23.766261 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:23.824731 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:23.824702 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:25.371167 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:25.371131 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:37.187156 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:37.187120 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:48:37.245839 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:48:37.245814 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:19.248814 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:19.248778 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:19.328315 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:19.328288 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:20.311747 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:20.311707 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:20.420497 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:20.420462 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:20.514697 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:20.514664 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:21.052798 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:21.052763 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:21.120666 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:21.120641 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:30.919049 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:30.919017 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:31.018055 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:31.018004 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:31.079139 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:31.079108 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:31.264862 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:31.264775 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:31.327905 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:31.327869 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:36.687896 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:36.687823 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:36.751850 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:36.751819 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:42.509470 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:42.509437 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:49:42.618201 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:49:42.618172 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:13.941904 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:13.941871 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 18:50:13.943422 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:13.943390 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 18:50:13.946324 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:13.946305 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:50:13.948082 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:13.948062 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:50:13.953540 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:13.953522 2587 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 18:50:20.531916 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:20.531878 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:21.868873 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:21.868835 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:51.788399 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:51.788351 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:52.776838 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:52.776801 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:55.423795 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:55.423753 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:55.514092 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:55.514062 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:56.484769 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:56.484738 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:56.585372 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:56.585342 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:56.863743 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:56.863668 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:56.959139 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:56.959111 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:59.291706 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:59.291608 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:50:59.499587 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:50:59.499552 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:01.902395 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:01.902362 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:02.002324 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:02.002291 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:02.182557 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:02.182470 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:02.247875 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:02.247844 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:06.485174 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:06.485092 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:10.521798 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:10.521761 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:10.597414 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:10.597382 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:11.810494 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:11.810462 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:11.910168 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:11.910106 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:15.044499 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:15.044466 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:15.114386 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:15.114347 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:18.044639 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:18.044608 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:18.172611 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:18.172576 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:19.942963 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:19.942913 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:23.713016 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:23.712976 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:31.803139 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:31.803107 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:31.908691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:31.908650 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:32.413960 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.413894 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:32.681473 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.681391 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:32.781130 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.781089 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:32.839728 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.839696 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt"] Jul 09 18:51:32.843082 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.843062 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:32.846288 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.846265 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 18:51:32.846402 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.846293 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 18:51:32.846402 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.846324 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 18:51:32.846617 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.846602 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 18:51:32.846842 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.846824 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9cq6g\"" Jul 09 18:51:32.857102 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.857084 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt"] Jul 09 18:51:32.912909 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.912890 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pjb7\" (UniqueName: \"kubernetes.io/projected/75e55877-5527-48d2-8709-729cc1f01eaf-kube-api-access-7pjb7\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:32.913027 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.912919 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75e55877-5527-48d2-8709-729cc1f01eaf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:32.913027 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.912999 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75e55877-5527-48d2-8709-729cc1f01eaf-manifests\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:32.913100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.913028 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75e55877-5527-48d2-8709-729cc1f01eaf-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:32.913100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.913058 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e55877-5527-48d2-8709-729cc1f01eaf-tmp\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:32.962578 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:32.962507 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:33.013395 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.013371 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75e55877-5527-48d2-8709-729cc1f01eaf-manifests\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.013486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.013399 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75e55877-5527-48d2-8709-729cc1f01eaf-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.013486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.013418 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e55877-5527-48d2-8709-729cc1f01eaf-tmp\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.013486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.013461 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pjb7\" (UniqueName: \"kubernetes.io/projected/75e55877-5527-48d2-8709-729cc1f01eaf-kube-api-access-7pjb7\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.013486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.013481 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75e55877-5527-48d2-8709-729cc1f01eaf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.013805 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.013787 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75e55877-5527-48d2-8709-729cc1f01eaf-manifests\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.015728 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.015704 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e55877-5527-48d2-8709-729cc1f01eaf-tmp\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.015851 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.015833 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75e55877-5527-48d2-8709-729cc1f01eaf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.016006 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.015989 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75e55877-5527-48d2-8709-729cc1f01eaf-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.023473 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.023450 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pjb7\" (UniqueName: \"kubernetes.io/projected/75e55877-5527-48d2-8709-729cc1f01eaf-kube-api-access-7pjb7\") pod \"opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt\" (UID: \"75e55877-5527-48d2-8709-729cc1f01eaf\") " pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.064637 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.064617 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:33.153390 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.153371 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:33.165259 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.165233 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:33.268672 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.268646 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:33.307362 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.307338 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt"] Jul 09 18:51:33.310438 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:51:33.310404 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e55877_5527_48d2_8709_729cc1f01eaf.slice/crio-456a45daa92470dcf2166221e0b8b225c41d9ab9e2118705fa2f3f94c1b078b7 WatchSource:0}: Error finding container 456a45daa92470dcf2166221e0b8b225c41d9ab9e2118705fa2f3f94c1b078b7: Status 404 returned error can't find the container with id 456a45daa92470dcf2166221e0b8b225c41d9ab9e2118705fa2f3f94c1b078b7 Jul 09 18:51:33.312156 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.312140 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:51:33.354381 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:33.354353 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:34.032845 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:34.032804 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" event={"ID":"75e55877-5527-48d2-8709-729cc1f01eaf","Type":"ContainerStarted","Data":"456a45daa92470dcf2166221e0b8b225c41d9ab9e2118705fa2f3f94c1b078b7"} Jul 09 18:51:37.047709 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:37.047681 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" event={"ID":"75e55877-5527-48d2-8709-729cc1f01eaf","Type":"ContainerStarted","Data":"9788069da0903611dc1ff7c57d3952c8a235b4ff39b9206c179838bfb1a9e384"} Jul 09 18:51:38.053022 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:38.052992 2587 generic.go:356] "Generic (PLEG): container finished" podID="75e55877-5527-48d2-8709-729cc1f01eaf" containerID="9788069da0903611dc1ff7c57d3952c8a235b4ff39b9206c179838bfb1a9e384" exitCode=0 Jul 09 18:51:38.053394 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:38.053054 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" event={"ID":"75e55877-5527-48d2-8709-729cc1f01eaf","Type":"ContainerDied","Data":"9788069da0903611dc1ff7c57d3952c8a235b4ff39b9206c179838bfb1a9e384"} Jul 09 18:51:38.148640 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:38.148608 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:38.254575 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:38.254535 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:39.058784 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:39.058748 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" event={"ID":"75e55877-5527-48d2-8709-729cc1f01eaf","Type":"ContainerStarted","Data":"ed25c1d987895b4c735af588115c11a79ee27e19d0bec364dfe17ed578fbcc41"} Jul 09 18:51:39.059172 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:39.058862 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:39.101442 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:39.101396 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" podStartSLOduration=2.797098686 podStartE2EDuration="7.101381467s" totalImagesPullingTime="3.244444858s" totalInitContainerRuntime="1.059837923s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:51:39.098999187 +0000 UTC m=+385.599958249" watchObservedRunningTime="2026-07-09 18:51:39.101381467 +0000 UTC m=+385.602340506" Jul 09 18:51:40.032706 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:40.032672 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:46.703320 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:46.703291 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:47.189835 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:47.189800 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:47.260277 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:47.260243 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:50.064548 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:50.064521 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt" Jul 09 18:51:51.919625 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:51.919597 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:52.026196 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.026163 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:52.670510 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.670467 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:52.827082 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.827052 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx"] Jul 09 18:51:52.835452 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.835426 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:52.838183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.838156 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 18:51:52.838373 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.838353 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c6rzh"] Jul 09 18:51:52.838491 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.838476 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 18:51:52.839422 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.839403 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 18:51:52.839490 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.839421 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mv78f\"" Jul 09 18:51:52.839490 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.839403 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 18:51:52.840318 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.840305 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:52.843409 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.843388 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 18:51:52.843490 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.843396 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-q5pkg\"" Jul 09 18:51:52.857117 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.857094 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx"] Jul 09 18:51:52.858469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.858450 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c6rzh"] Jul 09 18:51:52.931829 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.931732 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:52.965931 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.965904 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tmp\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:52.966077 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.965984 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tls-certs\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:52.966077 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.966020 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j96rj\" (UniqueName: \"kubernetes.io/projected/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-kube-api-access-j96rj\") pod \"odh-model-controller-7998c67dc4-c6rzh\" (UID: \"78afe730-cd3d-4ada-89a4-b2b54bdcbbda\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:52.966077 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.966037 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rdsx\" (UniqueName: \"kubernetes.io/projected/74c082ca-4fef-4f6e-8ef9-a474f53789fa-kube-api-access-9rdsx\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:52.966204 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:52.966164 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-cert\") pod \"odh-model-controller-7998c67dc4-c6rzh\" (UID: \"78afe730-cd3d-4ada-89a4-b2b54bdcbbda\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:53.024933 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.024899 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:53.066510 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.066484 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-cert\") pod \"odh-model-controller-7998c67dc4-c6rzh\" (UID: \"78afe730-cd3d-4ada-89a4-b2b54bdcbbda\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:53.066618 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.066525 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tmp\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.066618 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.066569 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tls-certs\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.066618 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:51:53.066581 2587 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 09 18:51:53.066618 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.066611 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j96rj\" (UniqueName: \"kubernetes.io/projected/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-kube-api-access-j96rj\") pod \"odh-model-controller-7998c67dc4-c6rzh\" (UID: \"78afe730-cd3d-4ada-89a4-b2b54bdcbbda\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:53.066808 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:51:53.066648 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-cert podName:78afe730-cd3d-4ada-89a4-b2b54bdcbbda nodeName:}" failed. No retries permitted until 2026-07-09 18:51:53.566626903 +0000 UTC m=+400.067585920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-cert") pod "odh-model-controller-7998c67dc4-c6rzh" (UID: "78afe730-cd3d-4ada-89a4-b2b54bdcbbda") : secret "odh-model-controller-webhook-cert" not found Jul 09 18:51:53.066808 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.066672 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rdsx\" (UniqueName: \"kubernetes.io/projected/74c082ca-4fef-4f6e-8ef9-a474f53789fa-kube-api-access-9rdsx\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.066808 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:51:53.066741 2587 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 09 18:51:53.066919 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:51:53.066808 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tls-certs podName:74c082ca-4fef-4f6e-8ef9-a474f53789fa nodeName:}" failed. No retries permitted until 2026-07-09 18:51:53.566789606 +0000 UTC m=+400.067748638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tls-certs") pod "kube-auth-proxy-86888fb94c-9vwhx" (UID: "74c082ca-4fef-4f6e-8ef9-a474f53789fa") : secret "kube-auth-proxy-tls" not found Jul 09 18:51:53.068757 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.068741 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tmp\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.084623 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.084594 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j96rj\" (UniqueName: \"kubernetes.io/projected/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-kube-api-access-j96rj\") pod \"odh-model-controller-7998c67dc4-c6rzh\" (UID: \"78afe730-cd3d-4ada-89a4-b2b54bdcbbda\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:53.085029 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.085012 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rdsx\" (UniqueName: \"kubernetes.io/projected/74c082ca-4fef-4f6e-8ef9-a474f53789fa-kube-api-access-9rdsx\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.124817 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.124797 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:53.227680 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.227584 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:53.343744 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.343715 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:53.534757 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.534676 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:53.571560 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.571537 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-cert\") pod \"odh-model-controller-7998c67dc4-c6rzh\" (UID: \"78afe730-cd3d-4ada-89a4-b2b54bdcbbda\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:53.571668 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.571581 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tls-certs\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.574074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.574049 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78afe730-cd3d-4ada-89a4-b2b54bdcbbda-cert\") pod \"odh-model-controller-7998c67dc4-c6rzh\" (UID: \"78afe730-cd3d-4ada-89a4-b2b54bdcbbda\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:53.574203 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.574128 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74c082ca-4fef-4f6e-8ef9-a474f53789fa-tls-certs\") pod \"kube-auth-proxy-86888fb94c-9vwhx\" (UID: \"74c082ca-4fef-4f6e-8ef9-a474f53789fa\") " pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.726002 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.725972 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rzjw2"] Jul 09 18:51:53.728183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.728168 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:53.732593 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.732573 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 18:51:53.732770 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.732703 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:53.746126 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.746110 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" Jul 09 18:51:53.750157 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.750122 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rzjw2"] Jul 09 18:51:53.752838 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.752654 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:51:53.873644 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.873614 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58n7\" (UniqueName: \"kubernetes.io/projected/48a521dd-49a5-4fd1-b6ea-624e4588db80-kube-api-access-c58n7\") pod \"model-serving-api-54d89bd5fc-rzjw2\" (UID: \"48a521dd-49a5-4fd1-b6ea-624e4588db80\") " pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:53.873788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.873655 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a521dd-49a5-4fd1-b6ea-624e4588db80-tls-certs\") pod \"model-serving-api-54d89bd5fc-rzjw2\" (UID: \"48a521dd-49a5-4fd1-b6ea-624e4588db80\") " pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:53.919286 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.919153 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx"] Jul 09 18:51:53.921303 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:51:53.921282 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c082ca_4fef_4f6e_8ef9_a474f53789fa.slice/crio-cc123eaf4dcaf2c6182ebe3bcba1fc26354a558b06b3bc80af7331226244b8a2 WatchSource:0}: Error finding container cc123eaf4dcaf2c6182ebe3bcba1fc26354a558b06b3bc80af7331226244b8a2: Status 404 returned error can't find the container with id cc123eaf4dcaf2c6182ebe3bcba1fc26354a558b06b3bc80af7331226244b8a2 Jul 09 18:51:53.937734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.937692 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c6rzh"] Jul 09 18:51:53.939650 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:51:53.939628 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78afe730_cd3d_4ada_89a4_b2b54bdcbbda.slice/crio-8451dac7d2fd477e743400270f4b178ce9dbe3fb11d1a385d3625d9fc3a65684 WatchSource:0}: Error finding container 8451dac7d2fd477e743400270f4b178ce9dbe3fb11d1a385d3625d9fc3a65684: Status 404 returned error can't find the container with id 8451dac7d2fd477e743400270f4b178ce9dbe3fb11d1a385d3625d9fc3a65684 Jul 09 18:51:53.974621 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.974594 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58n7\" (UniqueName: \"kubernetes.io/projected/48a521dd-49a5-4fd1-b6ea-624e4588db80-kube-api-access-c58n7\") pod \"model-serving-api-54d89bd5fc-rzjw2\" (UID: \"48a521dd-49a5-4fd1-b6ea-624e4588db80\") " pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:53.974748 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.974626 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a521dd-49a5-4fd1-b6ea-624e4588db80-tls-certs\") pod \"model-serving-api-54d89bd5fc-rzjw2\" (UID: \"48a521dd-49a5-4fd1-b6ea-624e4588db80\") " pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:53.974748 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:51:53.974738 2587 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 09 18:51:53.974864 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:51:53.974784 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a521dd-49a5-4fd1-b6ea-624e4588db80-tls-certs podName:48a521dd-49a5-4fd1-b6ea-624e4588db80 nodeName:}" failed. No retries permitted until 2026-07-09 18:51:54.474769812 +0000 UTC m=+400.975728827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/48a521dd-49a5-4fd1-b6ea-624e4588db80-tls-certs") pod "model-serving-api-54d89bd5fc-rzjw2" (UID: "48a521dd-49a5-4fd1-b6ea-624e4588db80") : secret "model-serving-api-tls" not found Jul 09 18:51:53.986088 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:53.986063 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58n7\" (UniqueName: \"kubernetes.io/projected/48a521dd-49a5-4fd1-b6ea-624e4588db80-kube-api-access-c58n7\") pod \"model-serving-api-54d89bd5fc-rzjw2\" (UID: \"48a521dd-49a5-4fd1-b6ea-624e4588db80\") " pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:54.068099 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.067971 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:54.117632 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.117595 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" event={"ID":"74c082ca-4fef-4f6e-8ef9-a474f53789fa","Type":"ContainerStarted","Data":"cc123eaf4dcaf2c6182ebe3bcba1fc26354a558b06b3bc80af7331226244b8a2"} Jul 09 18:51:54.118679 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.118653 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" event={"ID":"78afe730-cd3d-4ada-89a4-b2b54bdcbbda","Type":"ContainerStarted","Data":"8451dac7d2fd477e743400270f4b178ce9dbe3fb11d1a385d3625d9fc3a65684"} Jul 09 18:51:54.478166 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.478125 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a521dd-49a5-4fd1-b6ea-624e4588db80-tls-certs\") pod \"model-serving-api-54d89bd5fc-rzjw2\" (UID: \"48a521dd-49a5-4fd1-b6ea-624e4588db80\") " pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:54.480446 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.480424 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a521dd-49a5-4fd1-b6ea-624e4588db80-tls-certs\") pod \"model-serving-api-54d89bd5fc-rzjw2\" (UID: \"48a521dd-49a5-4fd1-b6ea-624e4588db80\") " pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:54.484584 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.484558 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz"] Jul 09 18:51:54.485230 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.485208 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:54.487678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.487657 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.490763 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.490743 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 18:51:54.490862 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.490829 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 18:51:54.491107 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.491093 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gx42g\"" Jul 09 18:51:54.491289 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.491271 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 18:51:54.491351 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.491338 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 18:51:54.503453 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.503433 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz"] Jul 09 18:51:54.578805 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.578754 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxz5d\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-kube-api-access-bxz5d\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.578976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.578818 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.578976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.578910 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-token\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.578976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.578959 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-cacerts\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.579153 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.579011 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.579153 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.579050 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-local-certs\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.579153 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.579080 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.584439 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.584418 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:54.638537 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.638504 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:51:54.679962 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.679916 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-local-certs\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.680149 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.680058 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.680149 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.680127 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxz5d\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-kube-api-access-bxz5d\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.680263 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.680169 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.680263 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.680222 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-token\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.680263 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.680246 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-cacerts\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.680422 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.680290 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.680984 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.680931 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.682956 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.682914 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-cacerts\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.683210 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.683191 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.683761 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.683740 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.684441 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.684403 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-local-certs\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.690410 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.690385 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxz5d\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-kube-api-access-bxz5d\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.691759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.691737 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-token\") pod \"istiod-openshift-gateway-7574459656-5dcxz\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.796383 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.796349 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:51:54.818666 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.818582 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rzjw2"] Jul 09 18:51:54.821832 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:51:54.821798 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a521dd_49a5_4fd1_b6ea_624e4588db80.slice/crio-090cfb011015c8957139553541bc03a5348b99e05e006735b2e3720aba9fcbbe WatchSource:0}: Error finding container 090cfb011015c8957139553541bc03a5348b99e05e006735b2e3720aba9fcbbe: Status 404 returned error can't find the container with id 090cfb011015c8957139553541bc03a5348b99e05e006735b2e3720aba9fcbbe Jul 09 18:51:54.968290 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:54.968215 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz"] Jul 09 18:51:54.971817 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:51:54.971787 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7304a1_2f2d_4fe8_84ba_e14b7b64c219.slice/crio-3f15b45916a16fd74da3db1baf5451b01a8b6816ce75565d38762d5b92be0521 WatchSource:0}: Error finding container 3f15b45916a16fd74da3db1baf5451b01a8b6816ce75565d38762d5b92be0521: Status 404 returned error can't find the container with id 3f15b45916a16fd74da3db1baf5451b01a8b6816ce75565d38762d5b92be0521 Jul 09 18:51:55.126833 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:55.126743 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" event={"ID":"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219","Type":"ContainerStarted","Data":"3f15b45916a16fd74da3db1baf5451b01a8b6816ce75565d38762d5b92be0521"} Jul 09 18:51:55.128107 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:55.128075 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" event={"ID":"48a521dd-49a5-4fd1-b6ea-624e4588db80","Type":"ContainerStarted","Data":"090cfb011015c8957139553541bc03a5348b99e05e006735b2e3720aba9fcbbe"} Jul 09 18:51:55.690231 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:55.690199 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:59.187742 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:59.187686 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:51:59.188144 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:59.187785 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:51:59.750429 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:59.750345 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:51:59.852662 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:51:59.852617 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:00.090488 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.090398 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:00.153193 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.153158 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" event={"ID":"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219","Type":"ContainerStarted","Data":"93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2"} Jul 09 18:52:00.153380 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.153220 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:52:00.154811 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.154785 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" event={"ID":"48a521dd-49a5-4fd1-b6ea-624e4588db80","Type":"ContainerStarted","Data":"03978b95f9ae26577ad1d59114e03b307e079711f21a60b59ead86bbd7d4299c"} Jul 09 18:52:00.154932 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.154882 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:52:00.155098 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.155075 2587 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5dcxz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 18:52:00.155197 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.155120 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" podUID="6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 18:52:00.156427 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.156406 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" event={"ID":"74c082ca-4fef-4f6e-8ef9-a474f53789fa","Type":"ContainerStarted","Data":"fa49c4fd60f72766ff061eb2a77cc42390f28e02f3f514e4403710737f3d3c23"} Jul 09 18:52:00.157712 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.157692 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" event={"ID":"78afe730-cd3d-4ada-89a4-b2b54bdcbbda","Type":"ContainerStarted","Data":"6fd0809e0a90c775c9724219f748ddddfd9f449b61c61277c71ce263771513cf"} Jul 09 18:52:00.157783 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.157762 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:52:00.184140 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.184116 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:00.260181 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.260134 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" podStartSLOduration=3.0103423 podStartE2EDuration="8.260119734s" totalImagesPullingTime="5.249777434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:00.259091618 +0000 UTC m=+406.760050655" watchObservedRunningTime="2026-07-09 18:52:00.260119734 +0000 UTC m=+406.761078771" Jul 09 18:52:00.260499 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.260208 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" podStartSLOduration=2.047289165 podStartE2EDuration="6.260202515s" totalImagesPullingTime="4.21291335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:00.230925846 +0000 UTC m=+406.731884885" watchObservedRunningTime="2026-07-09 18:52:00.260202515 +0000 UTC m=+406.761161553" Jul 09 18:52:00.302860 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.302814 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" podStartSLOduration=2.9401665340000003 podStartE2EDuration="7.302798794s" totalImagesPullingTime="4.36263226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:00.301607139 +0000 UTC m=+406.802566176" watchObservedRunningTime="2026-07-09 18:52:00.302798794 +0000 UTC m=+406.803757832" Jul 09 18:52:00.329631 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.329584 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86888fb94c-9vwhx" podStartSLOduration=3.058664459 podStartE2EDuration="8.329570053s" totalImagesPullingTime="5.270905594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:00.325018927 +0000 UTC m=+406.825977984" watchObservedRunningTime="2026-07-09 18:52:00.329570053 +0000 UTC m=+406.830529090" Jul 09 18:52:00.338681 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:00.338652 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:01.163211 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:01.163181 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:52:03.888256 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:03.888220 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:04.010329 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.010295 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:04.028405 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.028379 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zbtln"] Jul 09 18:52:04.031741 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.031726 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.034790 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.034762 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 18:52:04.035626 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.035602 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2v994\"" Jul 09 18:52:04.045010 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.044990 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zbtln"] Jul 09 18:52:04.048343 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.048323 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l"] Jul 09 18:52:04.051378 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.051363 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:04.054270 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.054251 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 18:52:04.054398 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.054377 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mbvbz\"" Jul 09 18:52:04.067405 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.067384 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l"] Jul 09 18:52:04.088234 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.088206 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:04.165048 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.165017 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktn4t\" (UniqueName: \"kubernetes.io/projected/f3b80d39-2997-47cb-ba8b-84c98e0e9100-kube-api-access-ktn4t\") pod \"kserve-controller-manager-d6f4b65b8-zbtln\" (UID: \"f3b80d39-2997-47cb-ba8b-84c98e0e9100\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.165216 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.165060 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:04.165216 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.165118 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3b80d39-2997-47cb-ba8b-84c98e0e9100-cert\") pod \"kserve-controller-manager-d6f4b65b8-zbtln\" (UID: \"f3b80d39-2997-47cb-ba8b-84c98e0e9100\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.165216 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.165152 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtqfj\" (UniqueName: \"kubernetes.io/projected/14deb11e-0a77-4a7a-82be-50f03c77564b-kube-api-access-jtqfj\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:04.266043 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.266014 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktn4t\" (UniqueName: \"kubernetes.io/projected/f3b80d39-2997-47cb-ba8b-84c98e0e9100-kube-api-access-ktn4t\") pod \"kserve-controller-manager-d6f4b65b8-zbtln\" (UID: \"f3b80d39-2997-47cb-ba8b-84c98e0e9100\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.266193 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.266052 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:04.266193 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.266078 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3b80d39-2997-47cb-ba8b-84c98e0e9100-cert\") pod \"kserve-controller-manager-d6f4b65b8-zbtln\" (UID: \"f3b80d39-2997-47cb-ba8b-84c98e0e9100\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.266193 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.266110 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtqfj\" (UniqueName: \"kubernetes.io/projected/14deb11e-0a77-4a7a-82be-50f03c77564b-kube-api-access-jtqfj\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:04.266193 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:04.266182 2587 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 09 18:52:04.266350 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:04.266239 2587 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 09 18:52:04.266350 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:04.266243 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert podName:14deb11e-0a77-4a7a-82be-50f03c77564b nodeName:}" failed. No retries permitted until 2026-07-09 18:52:04.766224264 +0000 UTC m=+411.267183299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert") pod "llmisvc-controller-manager-777c6bd7d5-dxk7l" (UID: "14deb11e-0a77-4a7a-82be-50f03c77564b") : secret "llmisvc-webhook-server-cert" not found Jul 09 18:52:04.266350 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:04.266298 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3b80d39-2997-47cb-ba8b-84c98e0e9100-cert podName:f3b80d39-2997-47cb-ba8b-84c98e0e9100 nodeName:}" failed. No retries permitted until 2026-07-09 18:52:04.766280857 +0000 UTC m=+411.267239887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f3b80d39-2997-47cb-ba8b-84c98e0e9100-cert") pod "kserve-controller-manager-d6f4b65b8-zbtln" (UID: "f3b80d39-2997-47cb-ba8b-84c98e0e9100") : secret "kserve-webhook-server-cert" not found Jul 09 18:52:04.282209 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.282176 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtqfj\" (UniqueName: \"kubernetes.io/projected/14deb11e-0a77-4a7a-82be-50f03c77564b-kube-api-access-jtqfj\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:04.283290 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.283268 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktn4t\" (UniqueName: \"kubernetes.io/projected/f3b80d39-2997-47cb-ba8b-84c98e0e9100-kube-api-access-ktn4t\") pod \"kserve-controller-manager-d6f4b65b8-zbtln\" (UID: \"f3b80d39-2997-47cb-ba8b-84c98e0e9100\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.466238 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.466168 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:04.771633 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.771557 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:04.771633 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.771594 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3b80d39-2997-47cb-ba8b-84c98e0e9100-cert\") pod \"kserve-controller-manager-d6f4b65b8-zbtln\" (UID: \"f3b80d39-2997-47cb-ba8b-84c98e0e9100\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.771806 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:04.771708 2587 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 09 18:52:04.771806 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:04.771767 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert podName:14deb11e-0a77-4a7a-82be-50f03c77564b nodeName:}" failed. No retries permitted until 2026-07-09 18:52:05.77175029 +0000 UTC m=+412.272709306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert") pod "llmisvc-controller-manager-777c6bd7d5-dxk7l" (UID: "14deb11e-0a77-4a7a-82be-50f03c77564b") : secret "llmisvc-webhook-server-cert" not found Jul 09 18:52:04.773718 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.773701 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3b80d39-2997-47cb-ba8b-84c98e0e9100-cert\") pod \"kserve-controller-manager-d6f4b65b8-zbtln\" (UID: \"f3b80d39-2997-47cb-ba8b-84c98e0e9100\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.942154 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.942120 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:04.955792 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:04.955764 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:05.072170 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.072137 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zbtln"] Jul 09 18:52:05.074718 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:52:05.074679 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b80d39_2997_47cb_ba8b_84c98e0e9100.slice/crio-4cbdd20f00452b9437b59e333461e7a0053105ec6b7e08b83466dd175a3e941c WatchSource:0}: Error finding container 4cbdd20f00452b9437b59e333461e7a0053105ec6b7e08b83466dd175a3e941c: Status 404 returned error can't find the container with id 4cbdd20f00452b9437b59e333461e7a0053105ec6b7e08b83466dd175a3e941c Jul 09 18:52:05.180545 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.180513 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" event={"ID":"f3b80d39-2997-47cb-ba8b-84c98e0e9100","Type":"ContainerStarted","Data":"4cbdd20f00452b9437b59e333461e7a0053105ec6b7e08b83466dd175a3e941c"} Jul 09 18:52:05.229006 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.228971 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:05.351391 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.351307 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:05.428298 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.428272 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:05.775588 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.775552 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:05.780346 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.780305 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:05.782683 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.782660 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14deb11e-0a77-4a7a-82be-50f03c77564b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dxk7l\" (UID: \"14deb11e-0a77-4a7a-82be-50f03c77564b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:05.843373 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.843348 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:05.860452 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:05.860428 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:06.011716 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.011691 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l"] Jul 09 18:52:06.012365 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:52:06.012342 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod14deb11e_0a77_4a7a_82be_50f03c77564b.slice/crio-8d2e5dde452de82df30bb5685ea56e0dacbf982eadf2df75207d4efe4bddc8a5 WatchSource:0}: Error finding container 8d2e5dde452de82df30bb5685ea56e0dacbf982eadf2df75207d4efe4bddc8a5: Status 404 returned error can't find the container with id 8d2e5dde452de82df30bb5685ea56e0dacbf982eadf2df75207d4efe4bddc8a5 Jul 09 18:52:06.186296 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.186260 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" event={"ID":"14deb11e-0a77-4a7a-82be-50f03c77564b","Type":"ContainerStarted","Data":"8d2e5dde452de82df30bb5685ea56e0dacbf982eadf2df75207d4efe4bddc8a5"} Jul 09 18:52:06.430636 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.430600 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz"] Jul 09 18:52:06.430881 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.430855 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" podUID="6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" containerName="discovery" containerID="cri-o://93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2" gracePeriod=30 Jul 09 18:52:06.678738 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.678713 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:52:06.688183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.688071 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-cacerts\") pod \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " Jul 09 18:52:06.688183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.688119 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-local-certs\") pod \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " Jul 09 18:52:06.688183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.688145 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-kubeconfig\") pod \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " Jul 09 18:52:06.688183 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.688185 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-token\") pod \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " Jul 09 18:52:06.688445 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.688249 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-dns-cert\") pod \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " Jul 09 18:52:06.688445 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.688300 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-ca-configmap\") pod \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " Jul 09 18:52:06.688445 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.688335 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxz5d\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-kube-api-access-bxz5d\") pod \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\" (UID: \"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219\") " Jul 09 18:52:06.689349 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.689300 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" (UID: "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:52:06.690862 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.690831 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-cacerts" (OuterVolumeSpecName: "cacerts") pod "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" (UID: "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:52:06.691063 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.690958 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" (UID: "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:52:06.691142 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.691084 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-local-certs" (OuterVolumeSpecName: "local-certs") pod "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" (UID: "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:52:06.691218 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.691196 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-kube-api-access-bxz5d" (OuterVolumeSpecName: "kube-api-access-bxz5d") pod "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" (UID: "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219"). InnerVolumeSpecName "kube-api-access-bxz5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:52:06.691275 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.691216 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" (UID: "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:52:06.691511 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.691483 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" (UID: "6f7304a1-2f2d-4fe8-84ba-e14b7b64c219"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:52:06.789664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.789596 2587 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-dns-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:06.789664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.789623 2587 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-csr-ca-configmap\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:06.789664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.789633 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxz5d\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-kube-api-access-bxz5d\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:06.789664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.789642 2587 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-cacerts\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:06.789664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.789651 2587 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-local-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:06.789664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.789660 2587 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-kubeconfig\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:06.789664 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:06.789667 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:07.198711 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.198495 2587 generic.go:356] "Generic (PLEG): container finished" podID="6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" containerID="93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2" exitCode=0 Jul 09 18:52:07.198711 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.198545 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" Jul 09 18:52:07.198711 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.198573 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" event={"ID":"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219","Type":"ContainerDied","Data":"93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2"} Jul 09 18:52:07.198711 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.198623 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz" event={"ID":"6f7304a1-2f2d-4fe8-84ba-e14b7b64c219","Type":"ContainerDied","Data":"3f15b45916a16fd74da3db1baf5451b01a8b6816ce75565d38762d5b92be0521"} Jul 09 18:52:07.198711 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.198653 2587 scope.go:117] "RemoveContainer" containerID="93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2" Jul 09 18:52:07.226655 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.226628 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz"] Jul 09 18:52:07.233723 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.233694 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5dcxz"] Jul 09 18:52:07.268141 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.268117 2587 scope.go:117] "RemoveContainer" containerID="93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2" Jul 09 18:52:07.268514 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:07.268493 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2\": container with ID starting with 93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2 not found: ID does not exist" containerID="93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2" Jul 09 18:52:07.268560 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:07.268523 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2"} err="failed to get container status \"93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2\": rpc error: code = NotFound desc = could not find container \"93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2\": container with ID starting with 93faa2b34da04dd2ce42c22d35ce24340fa5bdfcb569596e8695214c919eb3a2 not found: ID does not exist" Jul 09 18:52:08.065019 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:08.064992 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" path="/var/lib/kubelet/pods/6f7304a1-2f2d-4fe8-84ba-e14b7b64c219/volumes" Jul 09 18:52:09.210074 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:09.210044 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" event={"ID":"14deb11e-0a77-4a7a-82be-50f03c77564b","Type":"ContainerStarted","Data":"34017df5f512f93c1fa52aa61478e9983007556970711a7e99b812770b0d64e4"} Jul 09 18:52:09.210424 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:09.210119 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:09.211552 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:09.211529 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" event={"ID":"f3b80d39-2997-47cb-ba8b-84c98e0e9100","Type":"ContainerStarted","Data":"6e3fdbc6d018db269534352130445855aa50654bf26f50f2095c2aeb3880f613"} Jul 09 18:52:09.211626 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:09.211573 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:09.232780 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:09.232737 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" podStartSLOduration=2.1999587529999998 podStartE2EDuration="5.232725042s" totalImagesPullingTime="3.032766289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:09.231215765 +0000 UTC m=+415.732174802" watchObservedRunningTime="2026-07-09 18:52:09.232725042 +0000 UTC m=+415.733684080" Jul 09 18:52:09.259628 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:09.259588 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" podStartSLOduration=3.144954446 podStartE2EDuration="6.259575311s" totalImagesPullingTime="3.114620865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:09.25759938 +0000 UTC m=+415.758558418" watchObservedRunningTime="2026-07-09 18:52:09.259575311 +0000 UTC m=+415.760534348" Jul 09 18:52:11.164772 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:11.164743 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-c6rzh" Jul 09 18:52:11.166883 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:11.166856 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rzjw2" Jul 09 18:52:24.685820 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:24.685784 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:40.217967 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:40.217872 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dxk7l" Jul 09 18:52:40.220710 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:40.220693 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zbtln" Jul 09 18:52:53.403067 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.403031 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:53.502959 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.502917 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:53.536090 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.536059 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rrwd"] Jul 09 18:52:53.536385 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.536374 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" containerName="discovery" Jul 09 18:52:53.536426 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.536387 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" containerName="discovery" Jul 09 18:52:53.536462 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.536454 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7304a1-2f2d-4fe8-84ba-e14b7b64c219" containerName="discovery" Jul 09 18:52:53.540693 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.540668 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" Jul 09 18:52:53.543488 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.543464 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 18:52:53.543635 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.543563 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-4l7r5\"" Jul 09 18:52:53.543635 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.543582 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 18:52:53.550836 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.550814 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rrwd"] Jul 09 18:52:53.608275 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.608244 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:52:53.662509 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.662449 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfsqt\" (UniqueName: \"kubernetes.io/projected/b6b3d155-d1e7-4ed3-ade0-9e821501d986-kube-api-access-nfsqt\") pod \"kuadrant-operator-catalog-5rrwd\" (UID: \"b6b3d155-d1e7-4ed3-ade0-9e821501d986\") " pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" Jul 09 18:52:53.763178 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.763150 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfsqt\" (UniqueName: \"kubernetes.io/projected/b6b3d155-d1e7-4ed3-ade0-9e821501d986-kube-api-access-nfsqt\") pod \"kuadrant-operator-catalog-5rrwd\" (UID: \"b6b3d155-d1e7-4ed3-ade0-9e821501d986\") " pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" Jul 09 18:52:53.773425 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.773406 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfsqt\" (UniqueName: \"kubernetes.io/projected/b6b3d155-d1e7-4ed3-ade0-9e821501d986-kube-api-access-nfsqt\") pod \"kuadrant-operator-catalog-5rrwd\" (UID: \"b6b3d155-d1e7-4ed3-ade0-9e821501d986\") " pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" Jul 09 18:52:53.851591 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.851559 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" Jul 09 18:52:53.892868 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.892832 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rrwd"] Jul 09 18:52:53.977081 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:53.977056 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rrwd"] Jul 09 18:52:53.980893 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:52:53.980866 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b3d155_d1e7_4ed3_ade0_9e821501d986.slice/crio-0efeaa24b043f92e9c58ba6476a16f054b47cd6e9fa305794ca871e2e3e044bc WatchSource:0}: Error finding container 0efeaa24b043f92e9c58ba6476a16f054b47cd6e9fa305794ca871e2e3e044bc: Status 404 returned error can't find the container with id 0efeaa24b043f92e9c58ba6476a16f054b47cd6e9fa305794ca871e2e3e044bc Jul 09 18:52:54.113758 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.113731 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-24ptc"] Jul 09 18:52:54.116730 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.116708 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:52:54.134299 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.134278 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-24ptc"] Jul 09 18:52:54.268192 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.268109 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnf76\" (UniqueName: \"kubernetes.io/projected/875f2617-eb22-49cc-a8e8-049982697a03-kube-api-access-pnf76\") pod \"kuadrant-operator-catalog-24ptc\" (UID: \"875f2617-eb22-49cc-a8e8-049982697a03\") " pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:52:54.369439 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.369407 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnf76\" (UniqueName: \"kubernetes.io/projected/875f2617-eb22-49cc-a8e8-049982697a03-kube-api-access-pnf76\") pod \"kuadrant-operator-catalog-24ptc\" (UID: \"875f2617-eb22-49cc-a8e8-049982697a03\") " pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:52:54.380157 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.380130 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnf76\" (UniqueName: \"kubernetes.io/projected/875f2617-eb22-49cc-a8e8-049982697a03-kube-api-access-pnf76\") pod \"kuadrant-operator-catalog-24ptc\" (UID: \"875f2617-eb22-49cc-a8e8-049982697a03\") " pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:52:54.399263 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.399238 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" event={"ID":"b6b3d155-d1e7-4ed3-ade0-9e821501d986","Type":"ContainerStarted","Data":"0efeaa24b043f92e9c58ba6476a16f054b47cd6e9fa305794ca871e2e3e044bc"} Jul 09 18:52:54.426395 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.426377 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:52:54.567327 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:54.567302 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-24ptc"] Jul 09 18:52:54.571453 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:52:54.571426 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875f2617_eb22_49cc_a8e8_049982697a03.slice/crio-7cc07e592ce7c23269b99f9a9eccb53ffce357e2569ea0ad11e92b7134cab3e5 WatchSource:0}: Error finding container 7cc07e592ce7c23269b99f9a9eccb53ffce357e2569ea0ad11e92b7134cab3e5: Status 404 returned error can't find the container with id 7cc07e592ce7c23269b99f9a9eccb53ffce357e2569ea0ad11e92b7134cab3e5 Jul 09 18:52:55.407507 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:55.407469 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" event={"ID":"875f2617-eb22-49cc-a8e8-049982697a03","Type":"ContainerStarted","Data":"7cc07e592ce7c23269b99f9a9eccb53ffce357e2569ea0ad11e92b7134cab3e5"} Jul 09 18:52:56.412842 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.412795 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" event={"ID":"b6b3d155-d1e7-4ed3-ade0-9e821501d986","Type":"ContainerStarted","Data":"b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b"} Jul 09 18:52:56.413298 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.412808 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" podUID="b6b3d155-d1e7-4ed3-ade0-9e821501d986" containerName="registry-server" containerID="cri-o://b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b" gracePeriod=2 Jul 09 18:52:56.414378 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.414357 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" event={"ID":"875f2617-eb22-49cc-a8e8-049982697a03","Type":"ContainerStarted","Data":"afcd4cba47a12b71552d3713968337a9fc4728612ff50c71f268b9dba0a0c41f"} Jul 09 18:52:56.432485 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.432443 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" podStartSLOduration=1.306406655 podStartE2EDuration="3.432428973s" totalImagesPullingTime="2.126022318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:56.430389813 +0000 UTC m=+462.931348864" watchObservedRunningTime="2026-07-09 18:52:56.432428973 +0000 UTC m=+462.933388010" Jul 09 18:52:56.449659 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.449621 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" podStartSLOduration=0.910860713 podStartE2EDuration="2.449609632s" totalImagesPullingTime="1.538748919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:56.448978142 +0000 UTC m=+462.949937202" watchObservedRunningTime="2026-07-09 18:52:56.449609632 +0000 UTC m=+462.950568669" Jul 09 18:52:56.643574 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.643551 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" Jul 09 18:52:56.788717 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.788632 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfsqt\" (UniqueName: \"kubernetes.io/projected/b6b3d155-d1e7-4ed3-ade0-9e821501d986-kube-api-access-nfsqt\") pod \"b6b3d155-d1e7-4ed3-ade0-9e821501d986\" (UID: \"b6b3d155-d1e7-4ed3-ade0-9e821501d986\") " Jul 09 18:52:56.790794 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.790765 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6b3d155-d1e7-4ed3-ade0-9e821501d986-kube-api-access-nfsqt" (OuterVolumeSpecName: "kube-api-access-nfsqt") pod "b6b3d155-d1e7-4ed3-ade0-9e821501d986" (UID: "b6b3d155-d1e7-4ed3-ade0-9e821501d986"). InnerVolumeSpecName "kube-api-access-nfsqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:52:56.889658 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:56.889619 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfsqt\" (UniqueName: \"kubernetes.io/projected/b6b3d155-d1e7-4ed3-ade0-9e821501d986-kube-api-access-nfsqt\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:52:57.419643 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.419609 2587 generic.go:356] "Generic (PLEG): container finished" podID="b6b3d155-d1e7-4ed3-ade0-9e821501d986" containerID="b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b" exitCode=0 Jul 09 18:52:57.420036 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.419651 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" Jul 09 18:52:57.420036 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.419698 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" event={"ID":"b6b3d155-d1e7-4ed3-ade0-9e821501d986","Type":"ContainerDied","Data":"b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b"} Jul 09 18:52:57.420036 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.419747 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rrwd" event={"ID":"b6b3d155-d1e7-4ed3-ade0-9e821501d986","Type":"ContainerDied","Data":"0efeaa24b043f92e9c58ba6476a16f054b47cd6e9fa305794ca871e2e3e044bc"} Jul 09 18:52:57.420036 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.419776 2587 scope.go:117] "RemoveContainer" containerID="b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b" Jul 09 18:52:57.428353 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.428328 2587 scope.go:117] "RemoveContainer" containerID="b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b" Jul 09 18:52:57.428593 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:52:57.428578 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b\": container with ID starting with b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b not found: ID does not exist" containerID="b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b" Jul 09 18:52:57.428636 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.428601 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b"} err="failed to get container status \"b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b\": rpc error: code = NotFound desc = could not find container \"b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b\": container with ID starting with b12ca57598a9b8fc5500a4a8ccbe9745c2b095dde861e3ecb5602dd6cf82342b not found: ID does not exist" Jul 09 18:52:57.442719 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.442696 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rrwd"] Jul 09 18:52:57.447190 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:57.447168 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rrwd"] Jul 09 18:52:58.067678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:52:58.067641 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6b3d155-d1e7-4ed3-ade0-9e821501d986" path="/var/lib/kubelet/pods/b6b3d155-d1e7-4ed3-ade0-9e821501d986/volumes" Jul 09 18:53:04.427148 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:04.427105 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:53:04.427601 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:04.427170 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:53:04.454607 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:04.454582 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:53:04.476027 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:04.476002 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-24ptc" Jul 09 18:53:16.282331 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:16.282294 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:16.395265 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:16.395231 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:16.485431 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:16.485401 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:18.870653 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.870617 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh"] Jul 09 18:53:18.871047 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.870927 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6b3d155-d1e7-4ed3-ade0-9e821501d986" containerName="registry-server" Jul 09 18:53:18.871047 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.870937 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6b3d155-d1e7-4ed3-ade0-9e821501d986" containerName="registry-server" Jul 09 18:53:18.871047 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.871002 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6b3d155-d1e7-4ed3-ade0-9e821501d986" containerName="registry-server" Jul 09 18:53:18.874959 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.874928 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" Jul 09 18:53:18.877661 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.877635 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 18:53:18.877744 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.877714 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jcgm7\"" Jul 09 18:53:18.889032 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.889008 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh"] Jul 09 18:53:18.969639 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:18.969615 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mmvd\" (UniqueName: \"kubernetes.io/projected/ac456cf2-4b6f-4dd2-9aa9-16f34109a37b-kube-api-access-8mmvd\") pod \"dns-operator-controller-manager-864dd78c4c-wdvrh\" (UID: \"ac456cf2-4b6f-4dd2-9aa9-16f34109a37b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" Jul 09 18:53:19.070375 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:19.070342 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mmvd\" (UniqueName: \"kubernetes.io/projected/ac456cf2-4b6f-4dd2-9aa9-16f34109a37b-kube-api-access-8mmvd\") pod \"dns-operator-controller-manager-864dd78c4c-wdvrh\" (UID: \"ac456cf2-4b6f-4dd2-9aa9-16f34109a37b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" Jul 09 18:53:19.082888 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:19.082860 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mmvd\" (UniqueName: \"kubernetes.io/projected/ac456cf2-4b6f-4dd2-9aa9-16f34109a37b-kube-api-access-8mmvd\") pod \"dns-operator-controller-manager-864dd78c4c-wdvrh\" (UID: \"ac456cf2-4b6f-4dd2-9aa9-16f34109a37b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" Jul 09 18:53:19.185031 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:19.185006 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" Jul 09 18:53:19.322519 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:19.322495 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh"] Jul 09 18:53:19.325188 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:53:19.325159 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac456cf2_4b6f_4dd2_9aa9_16f34109a37b.slice/crio-8db3da442aed88eefc6314fcd6804910deda6f6811a038a2dd882ef2360f12ed WatchSource:0}: Error finding container 8db3da442aed88eefc6314fcd6804910deda6f6811a038a2dd882ef2360f12ed: Status 404 returned error can't find the container with id 8db3da442aed88eefc6314fcd6804910deda6f6811a038a2dd882ef2360f12ed Jul 09 18:53:19.517652 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:19.517574 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" event={"ID":"ac456cf2-4b6f-4dd2-9aa9-16f34109a37b","Type":"ContainerStarted","Data":"8db3da442aed88eefc6314fcd6804910deda6f6811a038a2dd882ef2360f12ed"} Jul 09 18:53:20.438593 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:20.438514 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:21.528636 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:21.528606 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" event={"ID":"ac456cf2-4b6f-4dd2-9aa9-16f34109a37b","Type":"ContainerStarted","Data":"8c92e8931b4c0e7f7549aac52d81491668ba4addd3c65622a5c8b27cbf49f172"} Jul 09 18:53:21.528988 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:21.528726 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" Jul 09 18:53:21.549958 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:21.549908 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" podStartSLOduration=1.474154963 podStartE2EDuration="3.549892667s" totalImagesPullingTime="2.075737704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:21.549249928 +0000 UTC m=+488.050209101" watchObservedRunningTime="2026-07-09 18:53:21.549892667 +0000 UTC m=+488.050851705" Jul 09 18:53:22.495663 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.495631 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69"] Jul 09 18:53:22.498929 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.498913 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:22.502898 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.502873 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t7cpt\"" Jul 09 18:53:22.510442 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.510423 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69"] Jul 09 18:53:22.599909 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.599871 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkdc7\" (UniqueName: \"kubernetes.io/projected/8d38fc61-7d2a-4f71-8bb3-e5984953e8a5-kube-api-access-fkdc7\") pod \"limitador-operator-controller-manager-859d7b78bd-rvg69\" (UID: \"8d38fc61-7d2a-4f71-8bb3-e5984953e8a5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:22.701229 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.701201 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkdc7\" (UniqueName: \"kubernetes.io/projected/8d38fc61-7d2a-4f71-8bb3-e5984953e8a5-kube-api-access-fkdc7\") pod \"limitador-operator-controller-manager-859d7b78bd-rvg69\" (UID: \"8d38fc61-7d2a-4f71-8bb3-e5984953e8a5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:22.710615 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.710589 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkdc7\" (UniqueName: \"kubernetes.io/projected/8d38fc61-7d2a-4f71-8bb3-e5984953e8a5-kube-api-access-fkdc7\") pod \"limitador-operator-controller-manager-859d7b78bd-rvg69\" (UID: \"8d38fc61-7d2a-4f71-8bb3-e5984953e8a5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:22.808786 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.808712 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:22.947294 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:22.947257 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69"] Jul 09 18:53:22.950228 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:53:22.950199 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d38fc61_7d2a_4f71_8bb3_e5984953e8a5.slice/crio-7efea4dbcae8cb002fb905f2913808c0b45dd37fa0b84fcb662e9451f591b4ba WatchSource:0}: Error finding container 7efea4dbcae8cb002fb905f2913808c0b45dd37fa0b84fcb662e9451f591b4ba: Status 404 returned error can't find the container with id 7efea4dbcae8cb002fb905f2913808c0b45dd37fa0b84fcb662e9451f591b4ba Jul 09 18:53:23.025437 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:23.025407 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:23.538161 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:23.538113 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" event={"ID":"8d38fc61-7d2a-4f71-8bb3-e5984953e8a5","Type":"ContainerStarted","Data":"7efea4dbcae8cb002fb905f2913808c0b45dd37fa0b84fcb662e9451f591b4ba"} Jul 09 18:53:23.639216 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:23.639187 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:25.548312 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:25.548278 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" event={"ID":"8d38fc61-7d2a-4f71-8bb3-e5984953e8a5","Type":"ContainerStarted","Data":"214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715"} Jul 09 18:53:25.548789 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:25.548353 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:25.576714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:25.576661 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" podStartSLOduration=1.535172274 podStartE2EDuration="3.576646305s" totalImagesPullingTime="2.041474031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:25.575703261 +0000 UTC m=+492.076662299" watchObservedRunningTime="2026-07-09 18:53:25.576646305 +0000 UTC m=+492.077605342" Jul 09 18:53:25.681854 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:25.681817 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:26.304127 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.304090 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7774t"] Jul 09 18:53:26.307449 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.307432 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" Jul 09 18:53:26.310103 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.310082 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5pt4d\"" Jul 09 18:53:26.317777 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.317750 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7774t"] Jul 09 18:53:26.434494 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.434465 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-876v4\" (UniqueName: \"kubernetes.io/projected/196641c8-2d05-4612-8365-30d9de32d76a-kube-api-access-876v4\") pod \"authorino-operator-75cc8d5cb7-7774t\" (UID: \"196641c8-2d05-4612-8365-30d9de32d76a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" Jul 09 18:53:26.535858 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.535825 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-876v4\" (UniqueName: \"kubernetes.io/projected/196641c8-2d05-4612-8365-30d9de32d76a-kube-api-access-876v4\") pod \"authorino-operator-75cc8d5cb7-7774t\" (UID: \"196641c8-2d05-4612-8365-30d9de32d76a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" Jul 09 18:53:26.546587 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.546561 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-876v4\" (UniqueName: \"kubernetes.io/projected/196641c8-2d05-4612-8365-30d9de32d76a-kube-api-access-876v4\") pod \"authorino-operator-75cc8d5cb7-7774t\" (UID: \"196641c8-2d05-4612-8365-30d9de32d76a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" Jul 09 18:53:26.618443 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.618360 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" Jul 09 18:53:26.753729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.753701 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7774t"] Jul 09 18:53:26.755772 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:53:26.755743 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196641c8_2d05_4612_8365_30d9de32d76a.slice/crio-b5ff32a8903cc377db2be46a88d8255a18cbe29e6a04d8364830b200acd8ce18 WatchSource:0}: Error finding container b5ff32a8903cc377db2be46a88d8255a18cbe29e6a04d8364830b200acd8ce18: Status 404 returned error can't find the container with id b5ff32a8903cc377db2be46a88d8255a18cbe29e6a04d8364830b200acd8ce18 Jul 09 18:53:26.860292 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:26.860260 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:27.028702 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:27.028670 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:27.168371 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:27.168324 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:27.272272 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:27.272242 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:27.509169 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:27.509137 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:27.559490 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:27.559461 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" event={"ID":"196641c8-2d05-4612-8365-30d9de32d76a","Type":"ContainerStarted","Data":"b5ff32a8903cc377db2be46a88d8255a18cbe29e6a04d8364830b200acd8ce18"} Jul 09 18:53:28.903112 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:28.903080 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:29.570760 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:29.570678 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" event={"ID":"196641c8-2d05-4612-8365-30d9de32d76a","Type":"ContainerStarted","Data":"f6fd71373e3f7eb556da20a16ac1ced03fa4ce4ee272a15c59bae69e04dad3dc"} Jul 09 18:53:29.570914 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:29.570809 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" Jul 09 18:53:29.600699 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:29.600651 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" podStartSLOduration=1.153984262 podStartE2EDuration="3.600639447s" totalImagesPullingTime="2.446655185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:29.599698482 +0000 UTC m=+496.100657524" watchObservedRunningTime="2026-07-09 18:53:29.600639447 +0000 UTC m=+496.101598485" Jul 09 18:53:31.067239 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.067209 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:31.750394 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.750352 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr"] Jul 09 18:53:31.754631 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.754602 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:31.758269 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.758252 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-btljp\"" Jul 09 18:53:31.782324 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.782295 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr"] Jul 09 18:53:31.880344 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.880311 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tb26\" (UniqueName: \"kubernetes.io/projected/4f1f4109-5536-4556-8644-b0979339dac9-kube-api-access-7tb26\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:31.880506 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.880374 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f1f4109-5536-4556-8644-b0979339dac9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:31.981764 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.981731 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tb26\" (UniqueName: \"kubernetes.io/projected/4f1f4109-5536-4556-8644-b0979339dac9-kube-api-access-7tb26\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:31.981911 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.981818 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f1f4109-5536-4556-8644-b0979339dac9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:31.982281 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.982262 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f1f4109-5536-4556-8644-b0979339dac9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:31.991763 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:31.991730 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tb26\" (UniqueName: \"kubernetes.io/projected/4f1f4109-5536-4556-8644-b0979339dac9-kube-api-access-7tb26\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:32.064361 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:32.064275 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:32.200833 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:32.200810 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr"] Jul 09 18:53:32.203029 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:53:32.203003 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1f4109_5536_4556_8644_b0979339dac9.slice/crio-120d7c1164289e5ddc5b7fbd617ba4413add109670a5938acd60744e8ff6e22a WatchSource:0}: Error finding container 120d7c1164289e5ddc5b7fbd617ba4413add109670a5938acd60744e8ff6e22a: Status 404 returned error can't find the container with id 120d7c1164289e5ddc5b7fbd617ba4413add109670a5938acd60744e8ff6e22a Jul 09 18:53:32.535957 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:32.535919 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wdvrh" Jul 09 18:53:32.585880 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:32.585845 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" event={"ID":"4f1f4109-5536-4556-8644-b0979339dac9","Type":"ContainerStarted","Data":"120d7c1164289e5ddc5b7fbd617ba4413add109670a5938acd60744e8ff6e22a"} Jul 09 18:53:33.268028 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:33.267992 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:33.380783 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:33.380753 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:34.921315 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:34.921273 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:35.021611 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:35.021582 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:36.555391 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:36.555362 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:36.679415 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:36.679380 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:36.780528 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:36.780492 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:36.879440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:36.879351 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:36.988443 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:36.988408 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:38.618251 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:38.618213 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" event={"ID":"4f1f4109-5536-4556-8644-b0979339dac9","Type":"ContainerStarted","Data":"91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d"} Jul 09 18:53:38.618699 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:38.618338 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:40.556652 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:40.556616 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:40.577017 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:40.576980 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7774t" Jul 09 18:53:40.603327 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:40.603279 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" podStartSLOduration=3.872946084 podStartE2EDuration="9.603265292s" totalImagesPullingTime="5.730319208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:38.647185741 +0000 UTC m=+505.148144780" watchObservedRunningTime="2026-07-09 18:53:40.603265292 +0000 UTC m=+507.104224329" Jul 09 18:53:40.651048 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:40.651007 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:41.243915 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:41.243881 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:41.344101 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:41.344071 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:41.449910 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:41.449882 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:41.549400 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:41.549322 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:45.219493 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:45.219466 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:45.328260 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:45.328231 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:49.245071 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:49.245038 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:49.345473 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:49.345442 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:49.625131 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:49.625057 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:50.889553 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:50.889521 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs"] Jul 09 18:53:50.892996 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:50.892978 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:50.911917 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:50.911896 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs"] Jul 09 18:53:51.027891 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.027857 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs"] Jul 09 18:53:51.028140 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.028121 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-6v49b], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" Jul 09 18:53:51.041541 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.041513 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v49b\" (UniqueName: \"kubernetes.io/projected/ed459f44-20b4-4194-bd9a-a52aa99afb32-kube-api-access-6v49b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.041658 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.041553 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed459f44-20b4-4194-bd9a-a52aa99afb32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.082823 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.082797 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7"] Jul 09 18:53:51.087502 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.087487 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.101310 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.101290 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7"] Jul 09 18:53:51.142362 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.142308 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v49b\" (UniqueName: \"kubernetes.io/projected/ed459f44-20b4-4194-bd9a-a52aa99afb32-kube-api-access-6v49b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.142362 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.142353 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed459f44-20b4-4194-bd9a-a52aa99afb32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.142670 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.142652 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed459f44-20b4-4194-bd9a-a52aa99afb32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.158147 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.158123 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v49b\" (UniqueName: \"kubernetes.io/projected/ed459f44-20b4-4194-bd9a-a52aa99afb32-kube-api-access-6v49b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.243131 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.243104 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.243269 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.243138 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvhmf\" (UniqueName: \"kubernetes.io/projected/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-kube-api-access-nvhmf\") pod \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.343497 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.343458 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.343497 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.343499 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhmf\" (UniqueName: \"kubernetes.io/projected/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-kube-api-access-nvhmf\") pod \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.343830 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.343813 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.358588 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.358566 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhmf\" (UniqueName: \"kubernetes.io/projected/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-kube-api-access-nvhmf\") pod \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.396492 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.396472 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:51.533588 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.533564 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7"] Jul 09 18:53:51.535779 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:53:51.535746 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0909d3e_a6a0_48cd_9902_9e15fa60d8c2.slice/crio-609c1bd992543e655e64eb44445a536721d81037d7b1e459094559790d53344b WatchSource:0}: Error finding container 609c1bd992543e655e64eb44445a536721d81037d7b1e459094559790d53344b: Status 404 returned error can't find the container with id 609c1bd992543e655e64eb44445a536721d81037d7b1e459094559790d53344b Jul 09 18:53:51.543874 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.543852 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7"] Jul 09 18:53:51.549001 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.548981 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7"] Jul 09 18:53:51.558960 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.558928 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs"] Jul 09 18:53:51.570160 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.569383 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs"] Jul 09 18:53:51.573358 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.573337 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2"] Jul 09 18:53:51.580570 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.580547 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr"] Jul 09 18:53:51.580690 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.580667 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:51.580752 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.580731 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" podUID="4f1f4109-5536-4556-8644-b0979339dac9" containerName="manager" containerID="cri-o://91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d" gracePeriod=2 Jul 09 18:53:51.591923 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.591898 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2"] Jul 09 18:53:51.604513 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.604465 2587 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"4f1f4109-5536-4556-8644-b0979339dac9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-x97gr\": pods \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.606818 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.606788 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="4f1f4109-5536-4556-8644-b0979339dac9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.607056 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.607028 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr"] Jul 09 18:53:51.613856 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.613833 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69"] Jul 09 18:53:51.614063 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.614043 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" containerName="manager" containerID="cri-o://214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715" gracePeriod=2 Jul 09 18:53:51.619712 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.619685 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk"] Jul 09 18:53:51.620146 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.620127 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f1f4109-5536-4556-8644-b0979339dac9" containerName="manager" Jul 09 18:53:51.620146 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.620147 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f1f4109-5536-4556-8644-b0979339dac9" containerName="manager" Jul 09 18:53:51.620291 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.620236 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f1f4109-5536-4556-8644-b0979339dac9" containerName="manager" Jul 09 18:53:51.624346 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.624174 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:51.625918 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.625891 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69"] Jul 09 18:53:51.641577 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.641555 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk"] Jul 09 18:53:51.644761 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.644740 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm48c\" (UniqueName: \"kubernetes.io/projected/7b6f63a1-0818-4fcb-8653-9ab335824ec6-kube-api-access-mm48c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wpmk\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:51.644920 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.644883 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b6f63a1-0818-4fcb-8653-9ab335824ec6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wpmk\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:51.663740 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.663702 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rvg69\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:51.665832 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.665802 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="4f1f4109-5536-4556-8644-b0979339dac9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.676901 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.676882 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.679418 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.679395 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.681663 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.681637 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rvg69\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:51.683682 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.683648 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="4f1f4109-5536-4556-8644-b0979339dac9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.705933 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.705917 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.708354 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.708332 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="4f1f4109-5536-4556-8644-b0979339dac9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.710386 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.710366 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.712750 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.712731 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rvg69\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:51.746540 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.746501 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f270850-c747-40b0-98cb-7569e0222daa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-274m2\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:51.746707 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.746565 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm48c\" (UniqueName: \"kubernetes.io/projected/7b6f63a1-0818-4fcb-8653-9ab335824ec6-kube-api-access-mm48c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wpmk\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:51.746707 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.746589 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mntz\" (UniqueName: \"kubernetes.io/projected/4f270850-c747-40b0-98cb-7569e0222daa-kube-api-access-5mntz\") pod \"kuadrant-operator-controller-manager-55467d9f7-274m2\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:51.746707 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.746645 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b6f63a1-0818-4fcb-8653-9ab335824ec6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wpmk\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:51.747013 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.746994 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b6f63a1-0818-4fcb-8653-9ab335824ec6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wpmk\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:51.757597 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.757572 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm48c\" (UniqueName: \"kubernetes.io/projected/7b6f63a1-0818-4fcb-8653-9ab335824ec6-kube-api-access-mm48c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wpmk\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:51.846957 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.846916 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed459f44-20b4-4194-bd9a-a52aa99afb32-extensions-socket-volume\") pod \"ed459f44-20b4-4194-bd9a-a52aa99afb32\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " Jul 09 18:53:51.847093 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.847013 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v49b\" (UniqueName: \"kubernetes.io/projected/ed459f44-20b4-4194-bd9a-a52aa99afb32-kube-api-access-6v49b\") pod \"ed459f44-20b4-4194-bd9a-a52aa99afb32\" (UID: \"ed459f44-20b4-4194-bd9a-a52aa99afb32\") " Jul 09 18:53:51.847206 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.847180 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed459f44-20b4-4194-bd9a-a52aa99afb32-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ed459f44-20b4-4194-bd9a-a52aa99afb32" (UID: "ed459f44-20b4-4194-bd9a-a52aa99afb32"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:53:51.847279 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.847197 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f270850-c747-40b0-98cb-7569e0222daa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-274m2\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:51.847279 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.847264 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mntz\" (UniqueName: \"kubernetes.io/projected/4f270850-c747-40b0-98cb-7569e0222daa-kube-api-access-5mntz\") pod \"kuadrant-operator-controller-manager-55467d9f7-274m2\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:51.847389 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.847373 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed459f44-20b4-4194-bd9a-a52aa99afb32-extensions-socket-volume\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:53:51.847543 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.847523 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f270850-c747-40b0-98cb-7569e0222daa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-274m2\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:51.849202 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.849182 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed459f44-20b4-4194-bd9a-a52aa99afb32-kube-api-access-6v49b" (OuterVolumeSpecName: "kube-api-access-6v49b") pod "ed459f44-20b4-4194-bd9a-a52aa99afb32" (UID: "ed459f44-20b4-4194-bd9a-a52aa99afb32"). InnerVolumeSpecName "kube-api-access-6v49b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:53:51.851339 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.851309 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.854410 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.854372 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.854915 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.854901 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:51.857061 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.857040 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rvg69\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:51.859185 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.859166 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="4f1f4109-5536-4556-8644-b0979339dac9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.861194 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.861178 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mntz\" (UniqueName: \"kubernetes.io/projected/4f270850-c747-40b0-98cb-7569e0222daa-kube-api-access-5mntz\") pod \"kuadrant-operator-controller-manager-55467d9f7-274m2\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:51.863075 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.863056 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:51.866210 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.866184 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rvg69\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:51.868591 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:51.868569 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-x97gr\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="4f1f4109-5536-4556-8644-b0979339dac9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:51.873400 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.873359 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:51.947837 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.947760 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f1f4109-5536-4556-8644-b0979339dac9-extensions-socket-volume\") pod \"4f1f4109-5536-4556-8644-b0979339dac9\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " Jul 09 18:53:51.947837 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.947802 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkdc7\" (UniqueName: \"kubernetes.io/projected/8d38fc61-7d2a-4f71-8bb3-e5984953e8a5-kube-api-access-fkdc7\") pod \"8d38fc61-7d2a-4f71-8bb3-e5984953e8a5\" (UID: \"8d38fc61-7d2a-4f71-8bb3-e5984953e8a5\") " Jul 09 18:53:51.948325 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.947843 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tb26\" (UniqueName: \"kubernetes.io/projected/4f1f4109-5536-4556-8644-b0979339dac9-kube-api-access-7tb26\") pod \"4f1f4109-5536-4556-8644-b0979339dac9\" (UID: \"4f1f4109-5536-4556-8644-b0979339dac9\") " Jul 09 18:53:51.948325 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.948001 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v49b\" (UniqueName: \"kubernetes.io/projected/ed459f44-20b4-4194-bd9a-a52aa99afb32-kube-api-access-6v49b\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:53:51.948411 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.948353 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f1f4109-5536-4556-8644-b0979339dac9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f1f4109-5536-4556-8644-b0979339dac9" (UID: "4f1f4109-5536-4556-8644-b0979339dac9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:53:51.949828 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.949806 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f1f4109-5536-4556-8644-b0979339dac9-kube-api-access-7tb26" (OuterVolumeSpecName: "kube-api-access-7tb26") pod "4f1f4109-5536-4556-8644-b0979339dac9" (UID: "4f1f4109-5536-4556-8644-b0979339dac9"). InnerVolumeSpecName "kube-api-access-7tb26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:53:51.949894 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.949875 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d38fc61-7d2a-4f71-8bb3-e5984953e8a5-kube-api-access-fkdc7" (OuterVolumeSpecName: "kube-api-access-fkdc7") pod "8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" (UID: "8d38fc61-7d2a-4f71-8bb3-e5984953e8a5"). InnerVolumeSpecName "kube-api-access-fkdc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:53:51.970195 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.970172 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:51.996087 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:51.996068 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:52.002914 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.002899 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:52.048449 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.048413 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f1f4109-5536-4556-8644-b0979339dac9-extensions-socket-volume\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:53:52.048563 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.048456 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkdc7\" (UniqueName: \"kubernetes.io/projected/8d38fc61-7d2a-4f71-8bb3-e5984953e8a5-kube-api-access-fkdc7\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:53:52.048563 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.048470 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tb26\" (UniqueName: \"kubernetes.io/projected/4f1f4109-5536-4556-8644-b0979339dac9-kube-api-access-7tb26\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:53:52.068540 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.068428 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f1f4109-5536-4556-8644-b0979339dac9" path="/var/lib/kubelet/pods/4f1f4109-5536-4556-8644-b0979339dac9/volumes" Jul 09 18:53:52.068935 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.068913 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" path="/var/lib/kubelet/pods/8d38fc61-7d2a-4f71-8bb3-e5984953e8a5/volumes" Jul 09 18:53:52.069407 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.069386 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" path="/var/lib/kubelet/pods/ed459f44-20b4-4194-bd9a-a52aa99afb32/volumes" Jul 09 18:53:52.146614 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.146581 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2"] Jul 09 18:53:52.151640 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:53:52.151610 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f270850_c747_40b0_98cb_7569e0222daa.slice/crio-ea0ad47ace558bf93f77253b493fd0796fc9ea283fb59811f8009e2dfccbaed6 WatchSource:0}: Error finding container ea0ad47ace558bf93f77253b493fd0796fc9ea283fb59811f8009e2dfccbaed6: Status 404 returned error can't find the container with id ea0ad47ace558bf93f77253b493fd0796fc9ea283fb59811f8009e2dfccbaed6 Jul 09 18:53:52.162674 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.162633 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk"] Jul 09 18:53:52.166207 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:53:52.166182 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b6f63a1_0818_4fcb_8653_9ab335824ec6.slice/crio-f0acec16bd01b5a46a7c6d148ff4f138f826d63fc73287509f4b8fa6957d96be WatchSource:0}: Error finding container f0acec16bd01b5a46a7c6d148ff4f138f826d63fc73287509f4b8fa6957d96be: Status 404 returned error can't find the container with id f0acec16bd01b5a46a7c6d148ff4f138f826d63fc73287509f4b8fa6957d96be Jul 09 18:53:52.682418 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.682381 2587 generic.go:356] "Generic (PLEG): container finished" podID="4f1f4109-5536-4556-8644-b0979339dac9" containerID="91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d" exitCode=0 Jul 09 18:53:52.682580 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.682428 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x97gr" Jul 09 18:53:52.682580 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.682476 2587 scope.go:117] "RemoveContainer" containerID="91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d" Jul 09 18:53:52.684218 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.684191 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" event={"ID":"4f270850-c747-40b0-98cb-7569e0222daa","Type":"ContainerStarted","Data":"042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8"} Jul 09 18:53:52.684335 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.684242 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" event={"ID":"4f270850-c747-40b0-98cb-7569e0222daa","Type":"ContainerStarted","Data":"ea0ad47ace558bf93f77253b493fd0796fc9ea283fb59811f8009e2dfccbaed6"} Jul 09 18:53:52.684335 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.684320 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:53:52.686086 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.686065 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" event={"ID":"7b6f63a1-0818-4fcb-8653-9ab335824ec6","Type":"ContainerStarted","Data":"669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd"} Jul 09 18:53:52.686212 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.686117 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" event={"ID":"7b6f63a1-0818-4fcb-8653-9ab335824ec6","Type":"ContainerStarted","Data":"f0acec16bd01b5a46a7c6d148ff4f138f826d63fc73287509f4b8fa6957d96be"} Jul 09 18:53:52.686212 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.686149 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:53:52.687379 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.687359 2587 generic.go:356] "Generic (PLEG): container finished" podID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" containerID="214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715" exitCode=0 Jul 09 18:53:52.687469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.687446 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rvg69" Jul 09 18:53:52.689079 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.689062 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:52.689260 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.689233 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" containerName="manager" containerID="cri-o://94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b" gracePeriod=2 Jul 09 18:53:52.692352 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.692337 2587 scope.go:117] "RemoveContainer" containerID="91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d" Jul 09 18:53:52.692590 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:52.692573 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d\": container with ID starting with 91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d not found: ID does not exist" containerID="91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d" Jul 09 18:53:52.692636 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.692595 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d"} err="failed to get container status \"91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d\": rpc error: code = NotFound desc = could not find container \"91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d\": container with ID starting with 91369051b0a95cf8c39e9052b76e119096604781e5f16ae95e511225f12c738d not found: ID does not exist" Jul 09 18:53:52.692636 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.692612 2587 scope.go:117] "RemoveContainer" containerID="214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715" Jul 09 18:53:52.700385 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.700368 2587 scope.go:117] "RemoveContainer" containerID="214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715" Jul 09 18:53:52.700613 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:52.700598 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715\": container with ID starting with 214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715 not found: ID does not exist" containerID="214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715" Jul 09 18:53:52.700671 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.700617 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715"} err="failed to get container status \"214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715\": rpc error: code = NotFound desc = could not find container \"214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715\": container with ID starting with 214409d7a028af002646c5dcc46140f21295b5c1609c3cb1327397e6d9dde715 not found: ID does not exist" Jul 09 18:53:52.716250 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.716211 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" podStartSLOduration=1.7161985400000002 podStartE2EDuration="1.71619854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:52.714880953 +0000 UTC m=+519.215839991" watchObservedRunningTime="2026-07-09 18:53:52.71619854 +0000 UTC m=+519.217157577" Jul 09 18:53:52.737930 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.737873 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" podStartSLOduration=1.737855926 podStartE2EDuration="1.737855926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:53:52.736291149 +0000 UTC m=+519.237250188" watchObservedRunningTime="2026-07-09 18:53:52.737855926 +0000 UTC m=+519.238814965" Jul 09 18:53:52.738589 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:52.738565 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:52.940250 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.940182 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:52.942703 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:52.942670 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:52.944717 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:52.944689 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:52.956545 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.956523 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-extensions-socket-volume\") pod \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " Jul 09 18:53:52.956837 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.956563 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvhmf\" (UniqueName: \"kubernetes.io/projected/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-kube-api-access-nvhmf\") pod \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\" (UID: \"a0909d3e-a6a0-48cd-9902-9e15fa60d8c2\") " Jul 09 18:53:52.956837 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.956784 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" (UID: "a0909d3e-a6a0-48cd-9902-9e15fa60d8c2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:53:52.958481 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:52.958461 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-kube-api-access-nvhmf" (OuterVolumeSpecName: "kube-api-access-nvhmf") pod "a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" (UID: "a0909d3e-a6a0-48cd-9902-9e15fa60d8c2"). InnerVolumeSpecName "kube-api-access-nvhmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:53:53.057131 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:53.057093 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-extensions-socket-volume\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:53:53.057131 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:53.057123 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvhmf\" (UniqueName: \"kubernetes.io/projected/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2-kube-api-access-nvhmf\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:53:53.697321 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:53.697285 2587 generic.go:356] "Generic (PLEG): container finished" podID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" containerID="94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b" exitCode=2 Jul 09 18:53:53.697321 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:53.697324 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:53.697520 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:53.697385 2587 scope.go:117] "RemoveContainer" containerID="94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b" Jul 09 18:53:53.700410 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:53.700374 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:53.702417 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:53.702386 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:53.706291 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:53.706271 2587 scope.go:117] "RemoveContainer" containerID="94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b" Jul 09 18:53:53.706549 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:53.706526 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b\": container with ID starting with 94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b not found: ID does not exist" containerID="94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b" Jul 09 18:53:53.706609 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:53.706556 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b"} err="failed to get container status \"94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b\": rpc error: code = NotFound desc = could not find container \"94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b\": container with ID starting with 94de45eedb0c95b1f8a548aa16c141b8936db9fb2cc28dae75378e0941bc7b3b not found: ID does not exist" Jul 09 18:53:53.708132 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:53.708104 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:53.710354 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:53.710333 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:54.065808 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:54.065716 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-77qr7\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-77qr7" Jul 09 18:53:54.066712 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:54.066690 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" path="/var/lib/kubelet/pods/a0909d3e-a6a0-48cd-9902-9e15fa60d8c2/volumes" Jul 09 18:53:54.068055 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:53:54.068025 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4cvxs\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="ed459f44-20b4-4194-bd9a-a52aa99afb32" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4cvxs" Jul 09 18:53:55.037492 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:55.037459 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:55.140690 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:55.140658 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:59.651141 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:59.651106 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:59.751162 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:59.751122 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:59.868931 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:59.868876 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:53:59.977083 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:53:59.976992 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:03.699755 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:03.699726 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:54:03.700245 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:03.699800 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:54:03.811033 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:03.811002 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk"] Jul 09 18:54:03.811234 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:03.811187 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" podUID="7b6f63a1-0818-4fcb-8653-9ab335824ec6" containerName="manager" containerID="cri-o://669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd" gracePeriod=10 Jul 09 18:54:04.059391 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.059368 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:54:04.132874 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.132848 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm48c\" (UniqueName: \"kubernetes.io/projected/7b6f63a1-0818-4fcb-8653-9ab335824ec6-kube-api-access-mm48c\") pod \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " Jul 09 18:54:04.133141 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.133123 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b6f63a1-0818-4fcb-8653-9ab335824ec6-extensions-socket-volume\") pod \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\" (UID: \"7b6f63a1-0818-4fcb-8653-9ab335824ec6\") " Jul 09 18:54:04.134714 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.134684 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b6f63a1-0818-4fcb-8653-9ab335824ec6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7b6f63a1-0818-4fcb-8653-9ab335824ec6" (UID: "7b6f63a1-0818-4fcb-8653-9ab335824ec6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:54:04.136369 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.136343 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b6f63a1-0818-4fcb-8653-9ab335824ec6-kube-api-access-mm48c" (OuterVolumeSpecName: "kube-api-access-mm48c") pod "7b6f63a1-0818-4fcb-8653-9ab335824ec6" (UID: "7b6f63a1-0818-4fcb-8653-9ab335824ec6"). InnerVolumeSpecName "kube-api-access-mm48c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:54:04.164644 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.164623 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5"] Jul 09 18:54:04.164981 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.164967 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" containerName="manager" Jul 09 18:54:04.165022 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.164985 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" containerName="manager" Jul 09 18:54:04.165022 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.165016 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" containerName="manager" Jul 09 18:54:04.165022 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.165022 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" containerName="manager" Jul 09 18:54:04.165117 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.165040 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b6f63a1-0818-4fcb-8653-9ab335824ec6" containerName="manager" Jul 09 18:54:04.165117 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.165046 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b6f63a1-0818-4fcb-8653-9ab335824ec6" containerName="manager" Jul 09 18:54:04.165117 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.165099 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b6f63a1-0818-4fcb-8653-9ab335824ec6" containerName="manager" Jul 09 18:54:04.165117 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.165106 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d38fc61-7d2a-4f71-8bb3-e5984953e8a5" containerName="manager" Jul 09 18:54:04.165117 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.165113 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0909d3e-a6a0-48cd-9902-9e15fa60d8c2" containerName="manager" Jul 09 18:54:04.168342 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.168328 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.188653 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.188626 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5"] Jul 09 18:54:04.234266 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.234199 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq8lk\" (UniqueName: \"kubernetes.io/projected/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-kube-api-access-rq8lk\") pod \"kuadrant-operator-controller-manager-86878c769f-zlzh5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.234266 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.234241 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zlzh5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.234408 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.234301 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b6f63a1-0818-4fcb-8653-9ab335824ec6-extensions-socket-volume\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:54:04.234408 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.234316 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm48c\" (UniqueName: \"kubernetes.io/projected/7b6f63a1-0818-4fcb-8653-9ab335824ec6-kube-api-access-mm48c\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:54:04.335314 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.335290 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq8lk\" (UniqueName: \"kubernetes.io/projected/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-kube-api-access-rq8lk\") pod \"kuadrant-operator-controller-manager-86878c769f-zlzh5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.335428 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.335320 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zlzh5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.335615 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.335599 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zlzh5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.353075 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.353055 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq8lk\" (UniqueName: \"kubernetes.io/projected/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-kube-api-access-rq8lk\") pod \"kuadrant-operator-controller-manager-86878c769f-zlzh5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.477729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.477704 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.615079 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.615055 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5"] Jul 09 18:54:04.616952 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:54:04.616914 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbb1b2d_2abf_4882_90db_4a41c24a8dc5.slice/crio-95a00395d8660a1960f9fb4be28ec688353fae694c80b184f134ce21c0021833 WatchSource:0}: Error finding container 95a00395d8660a1960f9fb4be28ec688353fae694c80b184f134ce21c0021833: Status 404 returned error can't find the container with id 95a00395d8660a1960f9fb4be28ec688353fae694c80b184f134ce21c0021833 Jul 09 18:54:04.751032 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.750954 2587 generic.go:356] "Generic (PLEG): container finished" podID="7b6f63a1-0818-4fcb-8653-9ab335824ec6" containerID="669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd" exitCode=0 Jul 09 18:54:04.751032 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.751010 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" Jul 09 18:54:04.751477 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.751034 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" event={"ID":"7b6f63a1-0818-4fcb-8653-9ab335824ec6","Type":"ContainerDied","Data":"669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd"} Jul 09 18:54:04.751477 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.751080 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk" event={"ID":"7b6f63a1-0818-4fcb-8653-9ab335824ec6","Type":"ContainerDied","Data":"f0acec16bd01b5a46a7c6d148ff4f138f826d63fc73287509f4b8fa6957d96be"} Jul 09 18:54:04.751477 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.751105 2587 scope.go:117] "RemoveContainer" containerID="669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd" Jul 09 18:54:04.752885 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.752855 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" event={"ID":"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5","Type":"ContainerStarted","Data":"2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760"} Jul 09 18:54:04.753022 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.752897 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" event={"ID":"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5","Type":"ContainerStarted","Data":"95a00395d8660a1960f9fb4be28ec688353fae694c80b184f134ce21c0021833"} Jul 09 18:54:04.753022 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.752962 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:04.760062 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.760047 2587 scope.go:117] "RemoveContainer" containerID="669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd" Jul 09 18:54:04.760301 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:54:04.760286 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd\": container with ID starting with 669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd not found: ID does not exist" containerID="669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd" Jul 09 18:54:04.760357 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.760309 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd"} err="failed to get container status \"669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd\": rpc error: code = NotFound desc = could not find container \"669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd\": container with ID starting with 669c559a1bee551cf0c88caa47c309b2b77c5f3638c2bc2db2881e32b4c1d2fd not found: ID does not exist" Jul 09 18:54:04.790345 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.790305 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" podStartSLOduration=0.790291249 podStartE2EDuration="790.291249ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:54:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:54:04.787507825 +0000 UTC m=+531.288466868" watchObservedRunningTime="2026-07-09 18:54:04.790291249 +0000 UTC m=+531.291250287" Jul 09 18:54:04.821654 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.821633 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk"] Jul 09 18:54:04.831317 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.831289 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wpmk"] Jul 09 18:54:04.874039 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:04.873997 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:05.006081 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:05.006003 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:06.065961 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:06.065917 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b6f63a1-0818-4fcb-8653-9ab335824ec6" path="/var/lib/kubelet/pods/7b6f63a1-0818-4fcb-8653-9ab335824ec6/volumes" Jul 09 18:54:15.766202 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:15.766167 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 18:54:15.841811 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:15.841783 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2"] Jul 09 18:54:15.842015 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:15.841979 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" podUID="4f270850-c747-40b0-98cb-7569e0222daa" containerName="manager" containerID="cri-o://042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8" gracePeriod=10 Jul 09 18:54:16.096563 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.096543 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:54:16.122217 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.122188 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f270850-c747-40b0-98cb-7569e0222daa-extensions-socket-volume\") pod \"4f270850-c747-40b0-98cb-7569e0222daa\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " Jul 09 18:54:16.122332 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.122222 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mntz\" (UniqueName: \"kubernetes.io/projected/4f270850-c747-40b0-98cb-7569e0222daa-kube-api-access-5mntz\") pod \"4f270850-c747-40b0-98cb-7569e0222daa\" (UID: \"4f270850-c747-40b0-98cb-7569e0222daa\") " Jul 09 18:54:16.122600 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.122573 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f270850-c747-40b0-98cb-7569e0222daa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f270850-c747-40b0-98cb-7569e0222daa" (UID: "4f270850-c747-40b0-98cb-7569e0222daa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:54:16.124625 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.124601 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f270850-c747-40b0-98cb-7569e0222daa-kube-api-access-5mntz" (OuterVolumeSpecName: "kube-api-access-5mntz") pod "4f270850-c747-40b0-98cb-7569e0222daa" (UID: "4f270850-c747-40b0-98cb-7569e0222daa"). InnerVolumeSpecName "kube-api-access-5mntz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:54:16.223686 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.223662 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f270850-c747-40b0-98cb-7569e0222daa-extensions-socket-volume\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:54:16.223686 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.223683 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mntz\" (UniqueName: \"kubernetes.io/projected/4f270850-c747-40b0-98cb-7569e0222daa-kube-api-access-5mntz\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:54:16.817993 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.817938 2587 generic.go:356] "Generic (PLEG): container finished" podID="4f270850-c747-40b0-98cb-7569e0222daa" containerID="042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8" exitCode=0 Jul 09 18:54:16.818430 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.818002 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" Jul 09 18:54:16.818430 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.818038 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" event={"ID":"4f270850-c747-40b0-98cb-7569e0222daa","Type":"ContainerDied","Data":"042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8"} Jul 09 18:54:16.818430 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.818105 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2" event={"ID":"4f270850-c747-40b0-98cb-7569e0222daa","Type":"ContainerDied","Data":"ea0ad47ace558bf93f77253b493fd0796fc9ea283fb59811f8009e2dfccbaed6"} Jul 09 18:54:16.818430 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.818140 2587 scope.go:117] "RemoveContainer" containerID="042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8" Jul 09 18:54:16.830592 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.830567 2587 scope.go:117] "RemoveContainer" containerID="042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8" Jul 09 18:54:16.830827 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:54:16.830809 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8\": container with ID starting with 042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8 not found: ID does not exist" containerID="042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8" Jul 09 18:54:16.830889 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.830834 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8"} err="failed to get container status \"042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8\": rpc error: code = NotFound desc = could not find container \"042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8\": container with ID starting with 042e543481c11a1440bd15527b2f6f921f88194e88b4dc4d04b46b861ff5c0c8 not found: ID does not exist" Jul 09 18:54:16.846927 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.846903 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2"] Jul 09 18:54:16.854114 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:16.854087 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-274m2"] Jul 09 18:54:18.065202 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:18.065173 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f270850-c747-40b0-98cb-7569e0222daa" path="/var/lib/kubelet/pods/4f270850-c747-40b0-98cb-7569e0222daa/volumes" Jul 09 18:54:20.337463 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.337430 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch"] Jul 09 18:54:20.337828 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.337791 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f270850-c747-40b0-98cb-7569e0222daa" containerName="manager" Jul 09 18:54:20.337828 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.337801 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f270850-c747-40b0-98cb-7569e0222daa" containerName="manager" Jul 09 18:54:20.337907 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.337864 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f270850-c747-40b0-98cb-7569e0222daa" containerName="manager" Jul 09 18:54:20.341166 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.341147 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.344027 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.344008 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 18:54:20.344232 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.344214 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xhcsn\"" Jul 09 18:54:20.353578 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.353554 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch"] Jul 09 18:54:20.361871 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.361850 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:20.455767 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455740 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.455897 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455771 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.455897 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455789 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.455897 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455822 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.455897 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455842 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.455897 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455863 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0225e57-c471-4ef4-ae65-f623a539d966-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.455897 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455879 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsx62\" (UniqueName: \"kubernetes.io/projected/e0225e57-c471-4ef4-ae65-f623a539d966-kube-api-access-vsx62\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.455897 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455897 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0225e57-c471-4ef4-ae65-f623a539d966-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.456132 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.455991 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0225e57-c471-4ef4-ae65-f623a539d966-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557138 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557115 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0225e57-c471-4ef4-ae65-f623a539d966-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557251 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557151 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557251 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557171 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557251 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557192 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557251 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557245 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557439 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557271 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557439 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557304 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0225e57-c471-4ef4-ae65-f623a539d966-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557439 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557329 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsx62\" (UniqueName: \"kubernetes.io/projected/e0225e57-c471-4ef4-ae65-f623a539d966-kube-api-access-vsx62\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557439 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557355 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0225e57-c471-4ef4-ae65-f623a539d966-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557605 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557678 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557634 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557777 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557710 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557927 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557895 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0225e57-c471-4ef4-ae65-f623a539d966-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.557927 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.557913 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.559629 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.559604 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0225e57-c471-4ef4-ae65-f623a539d966-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.559856 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.559834 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0225e57-c471-4ef4-ae65-f623a539d966-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.569394 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.569376 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0225e57-c471-4ef4-ae65-f623a539d966-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.569703 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.569685 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsx62\" (UniqueName: \"kubernetes.io/projected/e0225e57-c471-4ef4-ae65-f623a539d966-kube-api-access-vsx62\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hf5ch\" (UID: \"e0225e57-c471-4ef4-ae65-f623a539d966\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.651094 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.651058 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:20.660178 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.660154 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:20.788538 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.788511 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch"] Jul 09 18:54:20.790111 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:54:20.790076 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0225e57_c471_4ef4_ae65_f623a539d966.slice/crio-f041fdad57c93f4bb0ef4198ba0c869bc5db72ce2def186de7cc7f6c6e409b36 WatchSource:0}: Error finding container f041fdad57c93f4bb0ef4198ba0c869bc5db72ce2def186de7cc7f6c6e409b36: Status 404 returned error can't find the container with id f041fdad57c93f4bb0ef4198ba0c869bc5db72ce2def186de7cc7f6c6e409b36 Jul 09 18:54:20.839283 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:20.839260 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" event={"ID":"e0225e57-c471-4ef4-ae65-f623a539d966","Type":"ContainerStarted","Data":"f041fdad57c93f4bb0ef4198ba0c869bc5db72ce2def186de7cc7f6c6e409b36"} Jul 09 18:54:21.191743 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:21.191713 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:23.123203 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:23.123142 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:54:23.123435 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:23.123238 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:54:23.123435 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:23.123266 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:54:23.856658 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:23.856623 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" event={"ID":"e0225e57-c471-4ef4-ae65-f623a539d966","Type":"ContainerStarted","Data":"9d3ab1ed0a0d959eae116cbc91773383006ebb85b282b84f45c310aa7675cde4"} Jul 09 18:54:23.884272 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:23.884228 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" podStartSLOduration=1.5533390310000001 podStartE2EDuration="3.884213695s" totalImagesPullingTime="2.330874664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:54:23.881656507 +0000 UTC m=+550.382615549" watchObservedRunningTime="2026-07-09 18:54:23.884213695 +0000 UTC m=+550.385172810" Jul 09 18:54:24.651434 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:24.651396 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:24.656132 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:24.656109 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:24.862445 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:24.862417 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:24.863402 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:24.863381 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:54:26.407824 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:26.407795 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:33.787734 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.787695 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:33.802268 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.802231 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6pckg"] Jul 09 18:54:33.804640 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.804624 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:33.809226 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.809199 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4qkh2\"" Jul 09 18:54:33.809848 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.809828 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 18:54:33.831189 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.831160 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6pckg"] Jul 09 18:54:33.867659 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.867625 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-config-file\") pod \"limitador-limitador-64c94fd64f-6pckg\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:33.867830 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.867742 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rmbk\" (UniqueName: \"kubernetes.io/projected/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-kube-api-access-8rmbk\") pod \"limitador-limitador-64c94fd64f-6pckg\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:33.908502 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.908472 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6pckg"] Jul 09 18:54:33.929315 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.929289 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:33.968572 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.968542 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-config-file\") pod \"limitador-limitador-64c94fd64f-6pckg\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:33.968720 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.968589 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rmbk\" (UniqueName: \"kubernetes.io/projected/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-kube-api-access-8rmbk\") pod \"limitador-limitador-64c94fd64f-6pckg\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:33.969194 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.969175 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-config-file\") pod \"limitador-limitador-64c94fd64f-6pckg\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:33.982168 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:33.982143 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rmbk\" (UniqueName: \"kubernetes.io/projected/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-kube-api-access-8rmbk\") pod \"limitador-limitador-64c94fd64f-6pckg\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:34.007414 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.007386 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:34.115190 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.115103 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:34.276090 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.276061 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6pckg"] Jul 09 18:54:34.278084 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:54:34.278057 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9570505d_7e3a_48f7_a1f3_54b82bc1ed4c.slice/crio-cecd713c59727caf852d5552df135c2177ff90e8f4e91909306759310c8f3163 WatchSource:0}: Error finding container cecd713c59727caf852d5552df135c2177ff90e8f4e91909306759310c8f3163: Status 404 returned error can't find the container with id cecd713c59727caf852d5552df135c2177ff90e8f4e91909306759310c8f3163 Jul 09 18:54:34.781316 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.781284 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cd4hp"] Jul 09 18:54:34.784453 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.784425 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cd4hp" Jul 09 18:54:34.787389 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.787367 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-g5xxh\"" Jul 09 18:54:34.793667 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.793641 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cd4hp"] Jul 09 18:54:34.824052 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.824016 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:34.877522 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.877491 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lngzl\" (UniqueName: \"kubernetes.io/projected/87f8ddf5-cd34-40f5-a4b0-8060944ce4ef-kube-api-access-lngzl\") pod \"authorino-58cd955675-cd4hp\" (UID: \"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef\") " pod="kuadrant-system/authorino-58cd955675-cd4hp" Jul 09 18:54:34.910854 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.910776 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" event={"ID":"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c","Type":"ContainerStarted","Data":"cecd713c59727caf852d5552df135c2177ff90e8f4e91909306759310c8f3163"} Jul 09 18:54:34.923108 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.923038 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:34.978745 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.978712 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lngzl\" (UniqueName: \"kubernetes.io/projected/87f8ddf5-cd34-40f5-a4b0-8060944ce4ef-kube-api-access-lngzl\") pod \"authorino-58cd955675-cd4hp\" (UID: \"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef\") " pod="kuadrant-system/authorino-58cd955675-cd4hp" Jul 09 18:54:34.991256 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:34.991218 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lngzl\" (UniqueName: \"kubernetes.io/projected/87f8ddf5-cd34-40f5-a4b0-8060944ce4ef-kube-api-access-lngzl\") pod \"authorino-58cd955675-cd4hp\" (UID: \"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef\") " pod="kuadrant-system/authorino-58cd955675-cd4hp" Jul 09 18:54:35.095286 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:35.095206 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cd4hp" Jul 09 18:54:35.272254 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:35.272088 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cd4hp"] Jul 09 18:54:35.275118 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:54:35.275086 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87f8ddf5_cd34_40f5_a4b0_8060944ce4ef.slice/crio-523c71b991d001b00d7be3ceeb06f17d01690f20aab95c8add4579a8e56218bf WatchSource:0}: Error finding container 523c71b991d001b00d7be3ceeb06f17d01690f20aab95c8add4579a8e56218bf: Status 404 returned error can't find the container with id 523c71b991d001b00d7be3ceeb06f17d01690f20aab95c8add4579a8e56218bf Jul 09 18:54:35.918784 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:35.918720 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cd4hp" event={"ID":"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef","Type":"ContainerStarted","Data":"523c71b991d001b00d7be3ceeb06f17d01690f20aab95c8add4579a8e56218bf"} Jul 09 18:54:38.936044 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:38.936017 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" event={"ID":"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c","Type":"ContainerStarted","Data":"2227e7dd18e28078b118b17f3239847a12c88769af16a39ae86ec2df1700c996"} Jul 09 18:54:38.936345 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:38.936141 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:38.964403 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:38.964359 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" podStartSLOduration=1.379008258 podStartE2EDuration="5.964345011s" totalImagesPullingTime="4.585336753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:54:38.963526386 +0000 UTC m=+565.464485425" watchObservedRunningTime="2026-07-09 18:54:38.964345011 +0000 UTC m=+565.465304049" Jul 09 18:54:39.853269 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:39.853236 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cd4hp"] Jul 09 18:54:39.942602 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:39.942567 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cd4hp" event={"ID":"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef","Type":"ContainerStarted","Data":"5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590"} Jul 09 18:54:39.961003 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:39.960939 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-cd4hp" podStartSLOduration=2.314991322 podStartE2EDuration="5.960923756s" totalImagesPullingTime="3.645932434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:54:39.959820979 +0000 UTC m=+566.460780018" watchObservedRunningTime="2026-07-09 18:54:39.960923756 +0000 UTC m=+566.461882794" Jul 09 18:54:40.946906 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:40.946865 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-cd4hp" podUID="87f8ddf5-cd34-40f5-a4b0-8060944ce4ef" containerName="authorino" containerID="cri-o://5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590" gracePeriod=30 Jul 09 18:54:41.190123 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.190096 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cd4hp" Jul 09 18:54:41.336272 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.336187 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lngzl\" (UniqueName: \"kubernetes.io/projected/87f8ddf5-cd34-40f5-a4b0-8060944ce4ef-kube-api-access-lngzl\") pod \"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef\" (UID: \"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef\") " Jul 09 18:54:41.338218 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.338186 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87f8ddf5-cd34-40f5-a4b0-8060944ce4ef-kube-api-access-lngzl" (OuterVolumeSpecName: "kube-api-access-lngzl") pod "87f8ddf5-cd34-40f5-a4b0-8060944ce4ef" (UID: "87f8ddf5-cd34-40f5-a4b0-8060944ce4ef"). InnerVolumeSpecName "kube-api-access-lngzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:54:41.436922 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.436897 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lngzl\" (UniqueName: \"kubernetes.io/projected/87f8ddf5-cd34-40f5-a4b0-8060944ce4ef-kube-api-access-lngzl\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:54:41.953425 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.953391 2587 generic.go:356] "Generic (PLEG): container finished" podID="87f8ddf5-cd34-40f5-a4b0-8060944ce4ef" containerID="5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590" exitCode=0 Jul 09 18:54:41.953823 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.953433 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cd4hp" Jul 09 18:54:41.953823 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.953444 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cd4hp" event={"ID":"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef","Type":"ContainerDied","Data":"5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590"} Jul 09 18:54:41.953823 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.953476 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cd4hp" event={"ID":"87f8ddf5-cd34-40f5-a4b0-8060944ce4ef","Type":"ContainerDied","Data":"523c71b991d001b00d7be3ceeb06f17d01690f20aab95c8add4579a8e56218bf"} Jul 09 18:54:41.953823 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.953498 2587 scope.go:117] "RemoveContainer" containerID="5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590" Jul 09 18:54:41.961627 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.961610 2587 scope.go:117] "RemoveContainer" containerID="5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590" Jul 09 18:54:41.961880 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:54:41.961862 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590\": container with ID starting with 5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590 not found: ID does not exist" containerID="5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590" Jul 09 18:54:41.961928 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.961891 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590"} err="failed to get container status \"5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590\": rpc error: code = NotFound desc = could not find container \"5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590\": container with ID starting with 5f587c484d594924957635240310918b8bba2d7a9ccf267ae1803d61ad765590 not found: ID does not exist" Jul 09 18:54:41.975112 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.975089 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cd4hp"] Jul 09 18:54:41.978107 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:41.978079 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cd4hp"] Jul 09 18:54:42.065657 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:42.065626 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87f8ddf5-cd34-40f5-a4b0-8060944ce4ef" path="/var/lib/kubelet/pods/87f8ddf5-cd34-40f5-a4b0-8060944ce4ef/volumes" Jul 09 18:54:49.151034 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:49.150999 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6pckg"] Jul 09 18:54:49.151456 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:49.151156 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" podUID="9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" containerName="limitador" containerID="cri-o://2227e7dd18e28078b118b17f3239847a12c88769af16a39ae86ec2df1700c996" gracePeriod=30 Jul 09 18:54:49.151822 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:49.151804 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:49.991195 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:49.991164 2587 generic.go:356] "Generic (PLEG): container finished" podID="9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" containerID="2227e7dd18e28078b118b17f3239847a12c88769af16a39ae86ec2df1700c996" exitCode=0 Jul 09 18:54:49.991336 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:49.991210 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" event={"ID":"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c","Type":"ContainerDied","Data":"2227e7dd18e28078b118b17f3239847a12c88769af16a39ae86ec2df1700c996"} Jul 09 18:54:50.103095 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.103074 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:50.208685 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.208647 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-config-file\") pod \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " Jul 09 18:54:50.209096 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.208734 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rmbk\" (UniqueName: \"kubernetes.io/projected/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-kube-api-access-8rmbk\") pod \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\" (UID: \"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c\") " Jul 09 18:54:50.209096 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.208986 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-config-file" (OuterVolumeSpecName: "config-file") pod "9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" (UID: "9570505d-7e3a-48f7-a1f3-54b82bc1ed4c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:54:50.210753 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.210728 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-kube-api-access-8rmbk" (OuterVolumeSpecName: "kube-api-access-8rmbk") pod "9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" (UID: "9570505d-7e3a-48f7-a1f3-54b82bc1ed4c"). InnerVolumeSpecName "kube-api-access-8rmbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:54:50.310010 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.309979 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rmbk\" (UniqueName: \"kubernetes.io/projected/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-kube-api-access-8rmbk\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:54:50.310010 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.310008 2587 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c-config-file\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:54:50.996691 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.996658 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" event={"ID":"9570505d-7e3a-48f7-a1f3-54b82bc1ed4c","Type":"ContainerDied","Data":"cecd713c59727caf852d5552df135c2177ff90e8f4e91909306759310c8f3163"} Jul 09 18:54:50.996842 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.996703 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6pckg" Jul 09 18:54:50.996842 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:50.996740 2587 scope.go:117] "RemoveContainer" containerID="2227e7dd18e28078b118b17f3239847a12c88769af16a39ae86ec2df1700c996" Jul 09 18:54:51.021577 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.021552 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6pckg"] Jul 09 18:54:51.029788 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.029766 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6pckg"] Jul 09 18:54:51.317335 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.317255 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:51.413350 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.413319 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:51.814605 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.814565 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wv45g"] Jul 09 18:54:51.815080 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.815065 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" containerName="limitador" Jul 09 18:54:51.815123 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.815084 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" containerName="limitador" Jul 09 18:54:51.815123 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.815099 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87f8ddf5-cd34-40f5-a4b0-8060944ce4ef" containerName="authorino" Jul 09 18:54:51.815123 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.815107 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="87f8ddf5-cd34-40f5-a4b0-8060944ce4ef" containerName="authorino" Jul 09 18:54:51.815249 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.815238 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" containerName="limitador" Jul 09 18:54:51.815283 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.815256 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="87f8ddf5-cd34-40f5-a4b0-8060944ce4ef" containerName="authorino" Jul 09 18:54:51.820037 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.820017 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:51.822866 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.822820 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 18:54:51.822976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.822821 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 18:54:51.822976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.822901 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 18:54:51.824106 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.824086 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-v665d\"" Jul 09 18:54:51.828425 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.828405 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wv45g"] Jul 09 18:54:51.868977 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.868928 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:54:51.923426 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.923400 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/655e7b9b-669d-4ef8-bfbe-da9dbe696137-data\") pod \"postgres-69fb4594cf-wv45g\" (UID: \"655e7b9b-669d-4ef8-bfbe-da9dbe696137\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:51.923549 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:51.923507 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzbht\" (UniqueName: \"kubernetes.io/projected/655e7b9b-669d-4ef8-bfbe-da9dbe696137-kube-api-access-xzbht\") pod \"postgres-69fb4594cf-wv45g\" (UID: \"655e7b9b-669d-4ef8-bfbe-da9dbe696137\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:52.024362 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:52.024333 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzbht\" (UniqueName: \"kubernetes.io/projected/655e7b9b-669d-4ef8-bfbe-da9dbe696137-kube-api-access-xzbht\") pod \"postgres-69fb4594cf-wv45g\" (UID: \"655e7b9b-669d-4ef8-bfbe-da9dbe696137\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:52.024362 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:52.024364 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/655e7b9b-669d-4ef8-bfbe-da9dbe696137-data\") pod \"postgres-69fb4594cf-wv45g\" (UID: \"655e7b9b-669d-4ef8-bfbe-da9dbe696137\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:52.024723 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:52.024704 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/655e7b9b-669d-4ef8-bfbe-da9dbe696137-data\") pod \"postgres-69fb4594cf-wv45g\" (UID: \"655e7b9b-669d-4ef8-bfbe-da9dbe696137\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:52.037433 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:52.037411 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzbht\" (UniqueName: \"kubernetes.io/projected/655e7b9b-669d-4ef8-bfbe-da9dbe696137-kube-api-access-xzbht\") pod \"postgres-69fb4594cf-wv45g\" (UID: \"655e7b9b-669d-4ef8-bfbe-da9dbe696137\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:52.065177 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:52.065119 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9570505d-7e3a-48f7-a1f3-54b82bc1ed4c" path="/var/lib/kubelet/pods/9570505d-7e3a-48f7-a1f3-54b82bc1ed4c/volumes" Jul 09 18:54:52.130868 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:52.130849 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:52.265483 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:52.265460 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wv45g"] Jul 09 18:54:52.267350 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:54:52.267321 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655e7b9b_669d_4ef8_bfbe_da9dbe696137.slice/crio-9ea083bf655985c893066661378b8de3b43f2f096a2370c49ed0d9fd02e36cd1 WatchSource:0}: Error finding container 9ea083bf655985c893066661378b8de3b43f2f096a2370c49ed0d9fd02e36cd1: Status 404 returned error can't find the container with id 9ea083bf655985c893066661378b8de3b43f2f096a2370c49ed0d9fd02e36cd1 Jul 09 18:54:53.008706 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:53.008671 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" event={"ID":"655e7b9b-669d-4ef8-bfbe-da9dbe696137","Type":"ContainerStarted","Data":"9ea083bf655985c893066661378b8de3b43f2f096a2370c49ed0d9fd02e36cd1"} Jul 09 18:54:58.040283 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:58.040237 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" event={"ID":"655e7b9b-669d-4ef8-bfbe-da9dbe696137","Type":"ContainerStarted","Data":"7d63fb81635caf37c84dccb5075ff6aa41430104f8176894ed4e16c3265673bd"} Jul 09 18:54:58.040713 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:58.040304 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:54:58.064090 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:54:58.064046 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" podStartSLOduration=2.109224937 podStartE2EDuration="7.064031468s" totalImagesPullingTime="4.954806531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:54:58.062826021 +0000 UTC m=+584.563785059" watchObservedRunningTime="2026-07-09 18:54:58.064031468 +0000 UTC m=+584.564990505" Jul 09 18:55:04.074559 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.074531 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wv45g" Jul 09 18:55:04.410143 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.410112 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:04.661762 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.661685 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bvgpv"] Jul 09 18:55:04.665567 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.665547 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" Jul 09 18:55:04.668436 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.668417 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-g5xxh\"" Jul 09 18:55:04.673075 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.673052 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bvgpv"] Jul 09 18:55:04.733382 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.733356 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmrsc\" (UniqueName: \"kubernetes.io/projected/c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a-kube-api-access-wmrsc\") pod \"authorino-5d4b67c446-bvgpv\" (UID: \"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a\") " pod="kuadrant-system/authorino-5d4b67c446-bvgpv" Jul 09 18:55:04.742124 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.742098 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:04.834603 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.834579 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmrsc\" (UniqueName: \"kubernetes.io/projected/c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a-kube-api-access-wmrsc\") pod \"authorino-5d4b67c446-bvgpv\" (UID: \"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a\") " pod="kuadrant-system/authorino-5d4b67c446-bvgpv" Jul 09 18:55:04.845022 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.845000 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmrsc\" (UniqueName: \"kubernetes.io/projected/c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a-kube-api-access-wmrsc\") pod \"authorino-5d4b67c446-bvgpv\" (UID: \"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a\") " pod="kuadrant-system/authorino-5d4b67c446-bvgpv" Jul 09 18:55:04.898832 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.898807 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bvgpv"] Jul 09 18:55:04.899050 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.899039 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" Jul 09 18:55:04.932572 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.932504 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f95cb84f8-mrnst"] Jul 09 18:55:04.937284 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.937261 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95cb84f8-mrnst" Jul 09 18:55:04.944418 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:04.944394 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f95cb84f8-mrnst"] Jul 09 18:55:05.031910 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.031882 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bvgpv"] Jul 09 18:55:05.034389 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:55:05.034351 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9aa3f7f_660c_4cac_b5a3_a3d6710cbe4a.slice/crio-9d87efd788202bd151c77ab07cd7f46705dc8b689cc6bddea4aab7327aa9c97c WatchSource:0}: Error finding container 9d87efd788202bd151c77ab07cd7f46705dc8b689cc6bddea4aab7327aa9c97c: Status 404 returned error can't find the container with id 9d87efd788202bd151c77ab07cd7f46705dc8b689cc6bddea4aab7327aa9c97c Jul 09 18:55:05.035489 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.035466 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfcb4\" (UniqueName: \"kubernetes.io/projected/edea5a1e-4feb-4ae7-8671-3b91b3cf1026-kube-api-access-rfcb4\") pod \"authorino-6f95cb84f8-mrnst\" (UID: \"edea5a1e-4feb-4ae7-8671-3b91b3cf1026\") " pod="kuadrant-system/authorino-6f95cb84f8-mrnst" Jul 09 18:55:05.052356 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.052335 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f95cb84f8-mrnst"] Jul 09 18:55:05.052530 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:55:05.052515 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rfcb4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f95cb84f8-mrnst" podUID="edea5a1e-4feb-4ae7-8671-3b91b3cf1026" Jul 09 18:55:05.074976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.074934 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" event={"ID":"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a","Type":"ContainerStarted","Data":"9d87efd788202bd151c77ab07cd7f46705dc8b689cc6bddea4aab7327aa9c97c"} Jul 09 18:55:05.074976 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.074969 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95cb84f8-mrnst" Jul 09 18:55:05.082277 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.082259 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95cb84f8-mrnst" Jul 09 18:55:05.085931 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.085911 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67bc49c98-8m28h"] Jul 09 18:55:05.089071 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.089057 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.091903 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.091886 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 18:55:05.098324 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.098305 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67bc49c98-8m28h"] Jul 09 18:55:05.135890 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.135860 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfcb4\" (UniqueName: \"kubernetes.io/projected/edea5a1e-4feb-4ae7-8671-3b91b3cf1026-kube-api-access-rfcb4\") pod \"authorino-6f95cb84f8-mrnst\" (UID: \"edea5a1e-4feb-4ae7-8671-3b91b3cf1026\") " pod="kuadrant-system/authorino-6f95cb84f8-mrnst" Jul 09 18:55:05.145921 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.145890 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfcb4\" (UniqueName: \"kubernetes.io/projected/edea5a1e-4feb-4ae7-8671-3b91b3cf1026-kube-api-access-rfcb4\") pod \"authorino-6f95cb84f8-mrnst\" (UID: \"edea5a1e-4feb-4ae7-8671-3b91b3cf1026\") " pod="kuadrant-system/authorino-6f95cb84f8-mrnst" Jul 09 18:55:05.237100 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.237025 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfcb4\" (UniqueName: \"kubernetes.io/projected/edea5a1e-4feb-4ae7-8671-3b91b3cf1026-kube-api-access-rfcb4\") pod \"edea5a1e-4feb-4ae7-8671-3b91b3cf1026\" (UID: \"edea5a1e-4feb-4ae7-8671-3b91b3cf1026\") " Jul 09 18:55:05.237245 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.237116 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f2vz\" (UniqueName: \"kubernetes.io/projected/f7811595-763e-42d5-a766-2438b32eb639-kube-api-access-6f2vz\") pod \"authorino-67bc49c98-8m28h\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.237245 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.237150 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7811595-763e-42d5-a766-2438b32eb639-tls-cert\") pod \"authorino-67bc49c98-8m28h\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.238988 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.238965 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edea5a1e-4feb-4ae7-8671-3b91b3cf1026-kube-api-access-rfcb4" (OuterVolumeSpecName: "kube-api-access-rfcb4") pod "edea5a1e-4feb-4ae7-8671-3b91b3cf1026" (UID: "edea5a1e-4feb-4ae7-8671-3b91b3cf1026"). InnerVolumeSpecName "kube-api-access-rfcb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:55:05.338426 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.338389 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f2vz\" (UniqueName: \"kubernetes.io/projected/f7811595-763e-42d5-a766-2438b32eb639-kube-api-access-6f2vz\") pod \"authorino-67bc49c98-8m28h\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.338580 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.338437 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7811595-763e-42d5-a766-2438b32eb639-tls-cert\") pod \"authorino-67bc49c98-8m28h\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.338580 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.338487 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfcb4\" (UniqueName: \"kubernetes.io/projected/edea5a1e-4feb-4ae7-8671-3b91b3cf1026-kube-api-access-rfcb4\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:55:05.340863 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.340844 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7811595-763e-42d5-a766-2438b32eb639-tls-cert\") pod \"authorino-67bc49c98-8m28h\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.347974 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.347929 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f2vz\" (UniqueName: \"kubernetes.io/projected/f7811595-763e-42d5-a766-2438b32eb639-kube-api-access-6f2vz\") pod \"authorino-67bc49c98-8m28h\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.398801 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.398773 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:55:05.528425 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:05.528403 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67bc49c98-8m28h"] Jul 09 18:55:05.530061 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:55:05.530038 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7811595_763e_42d5_a766_2438b32eb639.slice/crio-8448c270e777458a1a1ce953caf62396186039be50634f3b13ad172f741eb1ca WatchSource:0}: Error finding container 8448c270e777458a1a1ce953caf62396186039be50634f3b13ad172f741eb1ca: Status 404 returned error can't find the container with id 8448c270e777458a1a1ce953caf62396186039be50634f3b13ad172f741eb1ca Jul 09 18:55:06.080640 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.080612 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" event={"ID":"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a","Type":"ContainerStarted","Data":"ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c"} Jul 09 18:55:06.081032 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.080666 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" podUID="c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a" containerName="authorino" containerID="cri-o://ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c" gracePeriod=30 Jul 09 18:55:06.081927 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.081904 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bc49c98-8m28h" event={"ID":"f7811595-763e-42d5-a766-2438b32eb639","Type":"ContainerStarted","Data":"8448c270e777458a1a1ce953caf62396186039be50634f3b13ad172f741eb1ca"} Jul 09 18:55:06.082045 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.081963 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95cb84f8-mrnst" Jul 09 18:55:06.101730 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.101690 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" podStartSLOduration=1.624223734 podStartE2EDuration="2.101678501s" totalImagesPullingTime="477.454767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:55:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:55:06.100104159 +0000 UTC m=+592.601063197" watchObservedRunningTime="2026-07-09 18:55:06.101678501 +0000 UTC m=+592.602637538" Jul 09 18:55:06.127993 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.127917 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f95cb84f8-mrnst"] Jul 09 18:55:06.133070 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.133045 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f95cb84f8-mrnst"] Jul 09 18:55:06.318966 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.318929 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" Jul 09 18:55:06.350771 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.350748 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmrsc\" (UniqueName: \"kubernetes.io/projected/c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a-kube-api-access-wmrsc\") pod \"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a\" (UID: \"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a\") " Jul 09 18:55:06.352701 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.352678 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a-kube-api-access-wmrsc" (OuterVolumeSpecName: "kube-api-access-wmrsc") pod "c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a" (UID: "c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a"). InnerVolumeSpecName "kube-api-access-wmrsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:55:06.451308 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:06.451283 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmrsc\" (UniqueName: \"kubernetes.io/projected/c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a-kube-api-access-wmrsc\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:55:07.087222 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.087186 2587 generic.go:356] "Generic (PLEG): container finished" podID="c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a" containerID="ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c" exitCode=0 Jul 09 18:55:07.087682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.087228 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" Jul 09 18:55:07.087682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.087259 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" event={"ID":"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a","Type":"ContainerDied","Data":"ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c"} Jul 09 18:55:07.087682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.087294 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-bvgpv" event={"ID":"c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a","Type":"ContainerDied","Data":"9d87efd788202bd151c77ab07cd7f46705dc8b689cc6bddea4aab7327aa9c97c"} Jul 09 18:55:07.087682 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.087317 2587 scope.go:117] "RemoveContainer" containerID="ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c" Jul 09 18:55:07.088688 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.088666 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bc49c98-8m28h" event={"ID":"f7811595-763e-42d5-a766-2438b32eb639","Type":"ContainerStarted","Data":"c751bae94f43c2896cf6c8aa58436d8be45cf00f7ba73bebdfd44806193606f7"} Jul 09 18:55:07.095867 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.095849 2587 scope.go:117] "RemoveContainer" containerID="ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c" Jul 09 18:55:07.096142 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:55:07.096126 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c\": container with ID starting with ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c not found: ID does not exist" containerID="ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c" Jul 09 18:55:07.096206 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.096151 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c"} err="failed to get container status \"ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c\": rpc error: code = NotFound desc = could not find container \"ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c\": container with ID starting with ce5d4041fc81eaa6fb1e6ed43d9eeb3d6ebeffd9dbcef27e7e29d9f2c9dd2b0c not found: ID does not exist" Jul 09 18:55:07.112613 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.112547 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67bc49c98-8m28h" podStartSLOduration=1.609745832 podStartE2EDuration="2.112534788s" totalImagesPullingTime="502.788956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:55:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:55:07.11028312 +0000 UTC m=+593.611242158" watchObservedRunningTime="2026-07-09 18:55:07.112534788 +0000 UTC m=+593.613493826" Jul 09 18:55:07.128683 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.128658 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bvgpv"] Jul 09 18:55:07.145444 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:07.145417 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bvgpv"] Jul 09 18:55:08.065757 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:08.065725 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a" path="/var/lib/kubelet/pods/c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a/volumes" Jul 09 18:55:08.066271 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:08.066251 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edea5a1e-4feb-4ae7-8671-3b91b3cf1026" path="/var/lib/kubelet/pods/edea5a1e-4feb-4ae7-8671-3b91b3cf1026/volumes" Jul 09 18:55:11.546416 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.546378 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:11.553844 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.553819 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d66988bd-wpg88"] Jul 09 18:55:11.554215 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.554193 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a" containerName="authorino" Jul 09 18:55:11.554215 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.554216 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a" containerName="authorino" Jul 09 18:55:11.554311 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.554271 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9aa3f7f-660c-4cac-b5a3-a3d6710cbe4a" containerName="authorino" Jul 09 18:55:11.598576 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.598541 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d66988bd-wpg88"] Jul 09 18:55:11.598736 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.598652 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:11.602059 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.602037 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 18:55:11.602179 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.602096 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-km6s9\"" Jul 09 18:55:11.603049 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.603025 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 18:55:11.624397 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.624373 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:11.695007 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.694976 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:11.696130 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.696111 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert\") pod \"maas-controller-6d66988bd-wpg88\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:11.696218 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.696152 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w5tq\" (UniqueName: \"kubernetes.io/projected/98ee265c-21c2-4d58-8aad-f39caaee1fb3-kube-api-access-7w5tq\") pod \"maas-controller-6d66988bd-wpg88\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:11.758463 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.758434 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-6vmvp"] Jul 09 18:55:11.792070 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.792033 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-6vmvp"] Jul 09 18:55:11.792212 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.792139 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:11.796738 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.796680 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert\") pod \"maas-controller-6d66988bd-wpg88\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:11.796850 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.796737 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w5tq\" (UniqueName: \"kubernetes.io/projected/98ee265c-21c2-4d58-8aad-f39caaee1fb3-kube-api-access-7w5tq\") pod \"maas-controller-6d66988bd-wpg88\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:11.797181 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:55:11.796849 2587 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 09 18:55:11.797181 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:55:11.796923 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert podName:98ee265c-21c2-4d58-8aad-f39caaee1fb3 nodeName:}" failed. No retries permitted until 2026-07-09 18:55:12.296901568 +0000 UTC m=+598.797860585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert") pod "maas-controller-6d66988bd-wpg88" (UID: "98ee265c-21c2-4d58-8aad-f39caaee1fb3") : secret "maas-controller-webhook-cert" not found Jul 09 18:55:11.806629 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.806608 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w5tq\" (UniqueName: \"kubernetes.io/projected/98ee265c-21c2-4d58-8aad-f39caaee1fb3-kube-api-access-7w5tq\") pod \"maas-controller-6d66988bd-wpg88\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:11.897773 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.897747 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b74a610-4340-4457-86e8-c504f937f95e-webhook-cert\") pod \"maas-controller-5b7f6c8fdb-6vmvp\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:11.897931 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.897802 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rghz\" (UniqueName: \"kubernetes.io/projected/6b74a610-4340-4457-86e8-c504f937f95e-kube-api-access-9rghz\") pod \"maas-controller-5b7f6c8fdb-6vmvp\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:11.935955 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.935914 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:11.998266 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.998244 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b74a610-4340-4457-86e8-c504f937f95e-webhook-cert\") pod \"maas-controller-5b7f6c8fdb-6vmvp\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:11.998371 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:11.998296 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rghz\" (UniqueName: \"kubernetes.io/projected/6b74a610-4340-4457-86e8-c504f937f95e-kube-api-access-9rghz\") pod \"maas-controller-5b7f6c8fdb-6vmvp\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:12.000588 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.000570 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b74a610-4340-4457-86e8-c504f937f95e-webhook-cert\") pod \"maas-controller-5b7f6c8fdb-6vmvp\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:12.010076 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.010055 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rghz\" (UniqueName: \"kubernetes.io/projected/6b74a610-4340-4457-86e8-c504f937f95e-kube-api-access-9rghz\") pod \"maas-controller-5b7f6c8fdb-6vmvp\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:12.102459 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.102379 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:12.259537 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.259510 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-6vmvp"] Jul 09 18:55:12.261605 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:55:12.261578 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b74a610_4340_4457_86e8_c504f937f95e.slice/crio-9a673c56434623154ba9cd17983baf964ce8e285c4665f32da2358707cf8541a WatchSource:0}: Error finding container 9a673c56434623154ba9cd17983baf964ce8e285c4665f32da2358707cf8541a: Status 404 returned error can't find the container with id 9a673c56434623154ba9cd17983baf964ce8e285c4665f32da2358707cf8541a Jul 09 18:55:12.301429 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.301395 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert\") pod \"maas-controller-6d66988bd-wpg88\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:12.303724 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.303699 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert\") pod \"maas-controller-6d66988bd-wpg88\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:12.510821 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.510787 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:12.674295 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:12.674267 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d66988bd-wpg88"] Jul 09 18:55:12.675784 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:55:12.675759 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ee265c_21c2_4d58_8aad_f39caaee1fb3.slice/crio-91e9d8e02a517daea3eec6771ed5cadea7b6de33a181efb912139948f56a023c WatchSource:0}: Error finding container 91e9d8e02a517daea3eec6771ed5cadea7b6de33a181efb912139948f56a023c: Status 404 returned error can't find the container with id 91e9d8e02a517daea3eec6771ed5cadea7b6de33a181efb912139948f56a023c Jul 09 18:55:13.118910 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:13.118868 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" event={"ID":"6b74a610-4340-4457-86e8-c504f937f95e","Type":"ContainerStarted","Data":"9a673c56434623154ba9cd17983baf964ce8e285c4665f32da2358707cf8541a"} Jul 09 18:55:13.120504 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:13.120469 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d66988bd-wpg88" event={"ID":"98ee265c-21c2-4d58-8aad-f39caaee1fb3","Type":"ContainerStarted","Data":"91e9d8e02a517daea3eec6771ed5cadea7b6de33a181efb912139948f56a023c"} Jul 09 18:55:13.987318 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:13.987290 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 18:55:13.988552 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:13.988475 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 18:55:13.995973 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:13.995935 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:55:13.996180 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:13.996165 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 18:55:15.234131 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:15.234105 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 18:55:15.433718 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:15.433690 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:15.533898 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:15.533820 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:16.137563 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:16.137523 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" event={"ID":"6b74a610-4340-4457-86e8-c504f937f95e","Type":"ContainerStarted","Data":"ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044"} Jul 09 18:55:16.137759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:16.137587 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:16.139182 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:16.139156 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d66988bd-wpg88" event={"ID":"98ee265c-21c2-4d58-8aad-f39caaee1fb3","Type":"ContainerStarted","Data":"92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420"} Jul 09 18:55:16.139287 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:16.139206 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:16.160995 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:16.160930 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" podStartSLOduration=2.19325722 podStartE2EDuration="5.160914238s" totalImagesPullingTime="2.967657018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:55:16.159119472 +0000 UTC m=+602.660078509" watchObservedRunningTime="2026-07-09 18:55:16.160914238 +0000 UTC m=+602.661873276" Jul 09 18:55:16.180440 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:16.180393 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d66988bd-wpg88" podStartSLOduration=2.6198836610000003 podStartE2EDuration="5.180378476s" totalImagesPullingTime="2.560494815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:55:16.177890328 +0000 UTC m=+602.678849387" watchObservedRunningTime="2026-07-09 18:55:16.180378476 +0000 UTC m=+602.681337546" Jul 09 18:55:27.152186 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.152155 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:27.153596 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.152412 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:55:27.224679 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.224650 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d66988bd-wpg88"] Jul 09 18:55:27.224924 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.224901 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d66988bd-wpg88" podUID="98ee265c-21c2-4d58-8aad-f39caaee1fb3" containerName="manager" containerID="cri-o://92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420" gracePeriod=30 Jul 09 18:55:27.471818 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.471796 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:27.635787 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.635754 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w5tq\" (UniqueName: \"kubernetes.io/projected/98ee265c-21c2-4d58-8aad-f39caaee1fb3-kube-api-access-7w5tq\") pod \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " Jul 09 18:55:27.635991 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.635837 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert\") pod \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\" (UID: \"98ee265c-21c2-4d58-8aad-f39caaee1fb3\") " Jul 09 18:55:27.637885 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.637860 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "98ee265c-21c2-4d58-8aad-f39caaee1fb3" (UID: "98ee265c-21c2-4d58-8aad-f39caaee1fb3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:55:27.638001 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.637932 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98ee265c-21c2-4d58-8aad-f39caaee1fb3-kube-api-access-7w5tq" (OuterVolumeSpecName: "kube-api-access-7w5tq") pod "98ee265c-21c2-4d58-8aad-f39caaee1fb3" (UID: "98ee265c-21c2-4d58-8aad-f39caaee1fb3"). InnerVolumeSpecName "kube-api-access-7w5tq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:55:27.736696 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.736601 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w5tq\" (UniqueName: \"kubernetes.io/projected/98ee265c-21c2-4d58-8aad-f39caaee1fb3-kube-api-access-7w5tq\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:55:27.736696 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:27.736639 2587 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98ee265c-21c2-4d58-8aad-f39caaee1fb3-webhook-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:55:28.193345 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.193312 2587 generic.go:356] "Generic (PLEG): container finished" podID="98ee265c-21c2-4d58-8aad-f39caaee1fb3" containerID="92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420" exitCode=0 Jul 09 18:55:28.193729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.193357 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d66988bd-wpg88" Jul 09 18:55:28.193729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.193402 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d66988bd-wpg88" event={"ID":"98ee265c-21c2-4d58-8aad-f39caaee1fb3","Type":"ContainerDied","Data":"92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420"} Jul 09 18:55:28.193729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.193453 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d66988bd-wpg88" event={"ID":"98ee265c-21c2-4d58-8aad-f39caaee1fb3","Type":"ContainerDied","Data":"91e9d8e02a517daea3eec6771ed5cadea7b6de33a181efb912139948f56a023c"} Jul 09 18:55:28.193729 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.193476 2587 scope.go:117] "RemoveContainer" containerID="92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420" Jul 09 18:55:28.201898 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.201883 2587 scope.go:117] "RemoveContainer" containerID="92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420" Jul 09 18:55:28.202171 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:55:28.202153 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420\": container with ID starting with 92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420 not found: ID does not exist" containerID="92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420" Jul 09 18:55:28.202264 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.202212 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420"} err="failed to get container status \"92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420\": rpc error: code = NotFound desc = could not find container \"92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420\": container with ID starting with 92ca93c369f60afc6960374d2dc6a8fa329dcabd1d8cbb4e56e9aac1357a1420 not found: ID does not exist" Jul 09 18:55:28.214383 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.214361 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d66988bd-wpg88"] Jul 09 18:55:28.219796 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:28.219771 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d66988bd-wpg88"] Jul 09 18:55:30.065685 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:30.065653 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98ee265c-21c2-4d58-8aad-f39caaee1fb3" path="/var/lib/kubelet/pods/98ee265c-21c2-4d58-8aad-f39caaee1fb3/volumes" Jul 09 18:55:46.999086 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:46.999003 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:48.197928 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.197896 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-4gsnx"] Jul 09 18:55:48.198332 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.198275 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98ee265c-21c2-4d58-8aad-f39caaee1fb3" containerName="manager" Jul 09 18:55:48.198332 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.198287 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="98ee265c-21c2-4d58-8aad-f39caaee1fb3" containerName="manager" Jul 09 18:55:48.198403 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.198336 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="98ee265c-21c2-4d58-8aad-f39caaee1fb3" containerName="manager" Jul 09 18:55:48.202737 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.202720 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.205545 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.205525 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 18:55:48.205674 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.205573 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsfbk\" (UniqueName: \"kubernetes.io/projected/bce1ad29-3d4a-479d-844b-a1b22d6ac880-kube-api-access-dsfbk\") pod \"payload-pre-processing-7964457696-4gsnx\" (UID: \"bce1ad29-3d4a-479d-844b-a1b22d6ac880\") " pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.205674 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.205609 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bce1ad29-3d4a-479d-844b-a1b22d6ac880-plugins-config-volume\") pod \"payload-pre-processing-7964457696-4gsnx\" (UID: \"bce1ad29-3d4a-479d-844b-a1b22d6ac880\") " pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.213211 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.213186 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-4gsnx"] Jul 09 18:55:48.221381 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.221354 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9lz4z"] Jul 09 18:55:48.224340 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.224317 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.226956 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.226924 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2lc24\"" Jul 09 18:55:48.233267 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.233245 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9lz4z"] Jul 09 18:55:48.306352 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.306317 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsfbk\" (UniqueName: \"kubernetes.io/projected/bce1ad29-3d4a-479d-844b-a1b22d6ac880-kube-api-access-dsfbk\") pod \"payload-pre-processing-7964457696-4gsnx\" (UID: \"bce1ad29-3d4a-479d-844b-a1b22d6ac880\") " pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.306536 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.306385 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bce1ad29-3d4a-479d-844b-a1b22d6ac880-plugins-config-volume\") pod \"payload-pre-processing-7964457696-4gsnx\" (UID: \"bce1ad29-3d4a-479d-844b-a1b22d6ac880\") " pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.306536 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.306439 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4b8280-2454-4da9-9cc5-a9a15587faf4-plugins-config-volume\") pod \"payload-processing-58f86c7798-9lz4z\" (UID: \"bf4b8280-2454-4da9-9cc5-a9a15587faf4\") " pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.306536 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.306468 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpmd4\" (UniqueName: \"kubernetes.io/projected/bf4b8280-2454-4da9-9cc5-a9a15587faf4-kube-api-access-rpmd4\") pod \"payload-processing-58f86c7798-9lz4z\" (UID: \"bf4b8280-2454-4da9-9cc5-a9a15587faf4\") " pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.307109 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.307086 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bce1ad29-3d4a-479d-844b-a1b22d6ac880-plugins-config-volume\") pod \"payload-pre-processing-7964457696-4gsnx\" (UID: \"bce1ad29-3d4a-479d-844b-a1b22d6ac880\") " pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.318377 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.318340 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsfbk\" (UniqueName: \"kubernetes.io/projected/bce1ad29-3d4a-479d-844b-a1b22d6ac880-kube-api-access-dsfbk\") pod \"payload-pre-processing-7964457696-4gsnx\" (UID: \"bce1ad29-3d4a-479d-844b-a1b22d6ac880\") " pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.407415 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.407385 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4b8280-2454-4da9-9cc5-a9a15587faf4-plugins-config-volume\") pod \"payload-processing-58f86c7798-9lz4z\" (UID: \"bf4b8280-2454-4da9-9cc5-a9a15587faf4\") " pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.407415 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.407417 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpmd4\" (UniqueName: \"kubernetes.io/projected/bf4b8280-2454-4da9-9cc5-a9a15587faf4-kube-api-access-rpmd4\") pod \"payload-processing-58f86c7798-9lz4z\" (UID: \"bf4b8280-2454-4da9-9cc5-a9a15587faf4\") " pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.408124 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.408104 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4b8280-2454-4da9-9cc5-a9a15587faf4-plugins-config-volume\") pod \"payload-processing-58f86c7798-9lz4z\" (UID: \"bf4b8280-2454-4da9-9cc5-a9a15587faf4\") " pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.417746 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.417715 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:55:48.418043 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.418026 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpmd4\" (UniqueName: \"kubernetes.io/projected/bf4b8280-2454-4da9-9cc5-a9a15587faf4-kube-api-access-rpmd4\") pod \"payload-processing-58f86c7798-9lz4z\" (UID: \"bf4b8280-2454-4da9-9cc5-a9a15587faf4\") " pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.513754 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.513671 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:48.537311 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.537284 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:48.666000 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.665907 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-4gsnx"] Jul 09 18:55:48.668201 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:55:48.668174 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce1ad29_3d4a_479d_844b_a1b22d6ac880.slice/crio-fc9f2b0221d63408ead1b9d6ffc3c642c1b22e87bf20dc3451da65fcecd2538d WatchSource:0}: Error finding container fc9f2b0221d63408ead1b9d6ffc3c642c1b22e87bf20dc3451da65fcecd2538d: Status 404 returned error can't find the container with id fc9f2b0221d63408ead1b9d6ffc3c642c1b22e87bf20dc3451da65fcecd2538d Jul 09 18:55:48.688781 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:48.688759 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-9lz4z"] Jul 09 18:55:48.691873 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:55:48.691846 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4b8280_2454_4da9_9cc5_a9a15587faf4.slice/crio-361d7909f462259921d8be374ce4789ba7c89e2f418f225ffb2472dc1c2ff4ec WatchSource:0}: Error finding container 361d7909f462259921d8be374ce4789ba7c89e2f418f225ffb2472dc1c2ff4ec: Status 404 returned error can't find the container with id 361d7909f462259921d8be374ce4789ba7c89e2f418f225ffb2472dc1c2ff4ec Jul 09 18:55:49.303202 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:49.303169 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" event={"ID":"bce1ad29-3d4a-479d-844b-a1b22d6ac880","Type":"ContainerStarted","Data":"fc9f2b0221d63408ead1b9d6ffc3c642c1b22e87bf20dc3451da65fcecd2538d"} Jul 09 18:55:49.304348 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:49.304324 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" event={"ID":"bf4b8280-2454-4da9-9cc5-a9a15587faf4","Type":"ContainerStarted","Data":"361d7909f462259921d8be374ce4789ba7c89e2f418f225ffb2472dc1c2ff4ec"} Jul 09 18:55:51.316040 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:51.316001 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" event={"ID":"bce1ad29-3d4a-479d-844b-a1b22d6ac880","Type":"ContainerStarted","Data":"78cb0eb33b6b74d2ca41568484f070a98201ff0f741bf749ab92f2595b6eabff"} Jul 09 18:55:51.316521 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:51.316095 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:55:51.317477 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:51.317444 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" event={"ID":"bf4b8280-2454-4da9-9cc5-a9a15587faf4","Type":"ContainerStarted","Data":"0d75d9e52ddf40c83b968c2f7b6fbfb74e50bdbe654250a7810da10b65ffa0d8"} Jul 09 18:55:51.317587 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:51.317522 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:55:51.341460 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:51.338514 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" podStartSLOduration=1.523570721 podStartE2EDuration="3.338498043s" totalImagesPullingTime="1.814927322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:55:51.337575173 +0000 UTC m=+637.838534228" watchObservedRunningTime="2026-07-09 18:55:51.338498043 +0000 UTC m=+637.839457082" Jul 09 18:55:55.904088 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:55.904048 2587 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hf5ch container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 18:55:55.905527 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:55.904127 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" podUID="e0225e57-c471-4ef4-ae65-f623a539d966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 18:55:56.342812 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:56.342715 2587 generic.go:356] "Generic (PLEG): container finished" podID="e0225e57-c471-4ef4-ae65-f623a539d966" containerID="9d3ab1ed0a0d959eae116cbc91773383006ebb85b282b84f45c310aa7675cde4" exitCode=0 Jul 09 18:55:56.342812 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:56.342789 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" event={"ID":"e0225e57-c471-4ef4-ae65-f623a539d966","Type":"ContainerDied","Data":"9d3ab1ed0a0d959eae116cbc91773383006ebb85b282b84f45c310aa7675cde4"} Jul 09 18:55:56.343185 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:56.343170 2587 scope.go:117] "RemoveContainer" containerID="9d3ab1ed0a0d959eae116cbc91773383006ebb85b282b84f45c310aa7675cde4" Jul 09 18:55:56.344252 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:56.344221 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:55:56.344319 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:56.344304 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:55:56.344366 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:56.344351 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 18:55:56.368141 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:56.368089 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" podStartSLOduration=6.574716664 podStartE2EDuration="8.368074002s" totalImagesPullingTime="1.793357338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:55:51.361696216 +0000 UTC m=+637.862655273" watchObservedRunningTime="2026-07-09 18:55:56.368074002 +0000 UTC m=+642.869033040" Jul 09 18:55:57.353191 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:57.353146 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" event={"ID":"e0225e57-c471-4ef4-ae65-f623a539d966","Type":"ContainerStarted","Data":"2c4e22964e74422a9bba2cdb235a918aaf1541e586e55242498c3093549b312a"} Jul 09 18:55:57.651933 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:57.651903 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:55:57.652287 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:57.652266 2587 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hf5ch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 09 18:55:57.652434 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:57.652409 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" podUID="e0225e57-c471-4ef4-ae65-f623a539d966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 09 18:55:58.652412 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:58.652376 2587 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hf5ch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 09 18:55:58.653015 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:58.652984 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" podUID="e0225e57-c471-4ef4-ae65-f623a539d966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 09 18:55:59.651579 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:59.651533 2587 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hf5ch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 09 18:55:59.651869 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:55:59.651837 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" podUID="e0225e57-c471-4ef4-ae65-f623a539d966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 09 18:56:00.652395 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:00.652353 2587 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hf5ch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 09 18:56:00.652879 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:00.652423 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" podUID="e0225e57-c471-4ef4-ae65-f623a539d966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 09 18:56:01.655235 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:01.655208 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:56:01.655598 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:01.655472 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:56:01.656242 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:01.656224 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hf5ch" Jul 09 18:56:02.324458 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:02.324428 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-4gsnx" Jul 09 18:56:02.324608 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:02.324493 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-9lz4z" Jul 09 18:56:03.188477 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:03.188447 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:56:03.445477 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:03.445396 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:56:09.587902 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.587859 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522"] Jul 09 18:56:09.591357 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.591336 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.594523 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.594500 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 18:56:09.594635 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.594548 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 18:56:09.595690 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.595670 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z4n5g\"" Jul 09 18:56:09.595690 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.595685 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 18:56:09.601328 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.601308 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522"] Jul 09 18:56:09.613784 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.613760 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:56:09.692586 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.692554 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.692746 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.692588 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5b7ca73-8b96-43c4-9995-cfd454087b6c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.692746 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.692630 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.692746 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.692718 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.692872 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.692784 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twmjx\" (UniqueName: \"kubernetes.io/projected/e5b7ca73-8b96-43c4-9995-cfd454087b6c-kube-api-access-twmjx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.692872 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.692811 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794051 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794017 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794230 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794067 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794230 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794122 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twmjx\" (UniqueName: \"kubernetes.io/projected/e5b7ca73-8b96-43c4-9995-cfd454087b6c-kube-api-access-twmjx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794230 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794154 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794230 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794228 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794454 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794250 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5b7ca73-8b96-43c4-9995-cfd454087b6c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794512 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794445 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794512 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794472 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.794620 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.794554 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.796373 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.796352 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5b7ca73-8b96-43c4-9995-cfd454087b6c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.796649 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.796630 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5b7ca73-8b96-43c4-9995-cfd454087b6c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.802916 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.802897 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twmjx\" (UniqueName: \"kubernetes.io/projected/e5b7ca73-8b96-43c4-9995-cfd454087b6c-kube-api-access-twmjx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522\" (UID: \"e5b7ca73-8b96-43c4-9995-cfd454087b6c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:09.902536 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:09.902502 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:10.043487 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:10.043460 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522"] Jul 09 18:56:10.045467 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:56:10.045429 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b7ca73_8b96_43c4_9995_cfd454087b6c.slice/crio-129bce0a0911fab37b103f48205a8bd319f343e3e76cfb506841965ce2213236 WatchSource:0}: Error finding container 129bce0a0911fab37b103f48205a8bd319f343e3e76cfb506841965ce2213236: Status 404 returned error can't find the container with id 129bce0a0911fab37b103f48205a8bd319f343e3e76cfb506841965ce2213236 Jul 09 18:56:10.428833 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:10.428781 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" event={"ID":"e5b7ca73-8b96-43c4-9995-cfd454087b6c","Type":"ContainerStarted","Data":"129bce0a0911fab37b103f48205a8bd319f343e3e76cfb506841965ce2213236"} Jul 09 18:56:11.437916 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:11.437877 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" event={"ID":"e5b7ca73-8b96-43c4-9995-cfd454087b6c","Type":"ContainerStarted","Data":"4afe4046741b0182195660fd769ed2ace505c37eee2cf94674ce823d062a1479"} Jul 09 18:56:11.460467 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:11.460419 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" podStartSLOduration=1.190725215 podStartE2EDuration="2.460403703s" totalImagesPullingTime="1.269678488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:56:11.458805989 +0000 UTC m=+657.959765029" watchObservedRunningTime="2026-07-09 18:56:11.460403703 +0000 UTC m=+657.961362740" Jul 09 18:56:19.902870 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:19.902833 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:19.903421 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:19.902889 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:19.915522 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:19.915502 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:20.490428 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:20.490396 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522" Jul 09 18:56:28.403698 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.403663 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4"] Jul 09 18:56:28.409390 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.409369 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.412800 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.412782 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 18:56:28.427422 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.427398 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4"] Jul 09 18:56:28.457219 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.457193 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:56:28.552357 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.552323 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.552541 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.552393 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.552541 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.552445 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.552671 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.552539 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.552671 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.552595 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.552671 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.552655 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t6mk\" (UniqueName: \"kubernetes.io/projected/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-kube-api-access-7t6mk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654017 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.653907 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654017 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.653986 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654252 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.654023 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t6mk\" (UniqueName: \"kubernetes.io/projected/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-kube-api-access-7t6mk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654252 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.654067 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654252 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.654113 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654252 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.654135 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654469 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.654408 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654529 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.654484 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.654529 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.654517 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.656207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.656186 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.656528 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.656509 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.675695 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.675673 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t6mk\" (UniqueName: \"kubernetes.io/projected/c17e81ba-4053-4f5d-afb8-fc0200d2c5ce-kube-api-access-7t6mk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wkgp4\" (UID: \"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.719647 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.719613 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:28.868701 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:28.868672 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4"] Jul 09 18:56:28.869786 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:56:28.869761 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17e81ba_4053_4f5d_afb8_fc0200d2c5ce.slice/crio-324c3a3448df0452b05db6a41ec147798574750ceeb5d655086b17aca93772d0 WatchSource:0}: Error finding container 324c3a3448df0452b05db6a41ec147798574750ceeb5d655086b17aca93772d0: Status 404 returned error can't find the container with id 324c3a3448df0452b05db6a41ec147798574750ceeb5d655086b17aca93772d0 Jul 09 18:56:29.522241 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:29.522196 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" event={"ID":"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce","Type":"ContainerStarted","Data":"3618383f9be9257d2c0b07c25fd0bcd888d5c22b9a3a9e97bc22655a24455763"} Jul 09 18:56:29.522742 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:29.522293 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" event={"ID":"c17e81ba-4053-4f5d-afb8-fc0200d2c5ce","Type":"ContainerStarted","Data":"324c3a3448df0452b05db6a41ec147798574750ceeb5d655086b17aca93772d0"} Jul 09 18:56:29.553525 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:29.553475 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" podStartSLOduration=1.3753983 podStartE2EDuration="1.553461148s" totalImagesPullingTime="178.062848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:56:29.550390375 +0000 UTC m=+676.051349453" watchObservedRunningTime="2026-07-09 18:56:29.553461148 +0000 UTC m=+676.054420182" Jul 09 18:56:34.897862 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:34.897825 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q"] Jul 09 18:56:34.901418 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:34.901402 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:34.904643 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:34.904622 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 18:56:34.911971 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:34.911929 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q"] Jul 09 18:56:34.916558 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:34.916531 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:56:35.013391 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.013356 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.013571 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.013412 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hklk8\" (UniqueName: \"kubernetes.io/projected/999cefec-d3db-47b2-9e0f-05cce59d30d5-kube-api-access-hklk8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.013571 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.013453 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.013571 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.013507 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.013571 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.013547 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.013571 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.013569 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/999cefec-d3db-47b2-9e0f-05cce59d30d5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114259 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114215 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114443 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114359 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114443 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114386 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/999cefec-d3db-47b2-9e0f-05cce59d30d5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114566 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114444 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114566 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114498 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hklk8\" (UniqueName: \"kubernetes.io/projected/999cefec-d3db-47b2-9e0f-05cce59d30d5-kube-api-access-hklk8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114566 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114547 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114769 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114745 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114832 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114791 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.114909 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.114892 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.116500 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.116481 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/999cefec-d3db-47b2-9e0f-05cce59d30d5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.116817 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.116800 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/999cefec-d3db-47b2-9e0f-05cce59d30d5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.124882 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.124859 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hklk8\" (UniqueName: \"kubernetes.io/projected/999cefec-d3db-47b2-9e0f-05cce59d30d5-kube-api-access-hklk8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-6m25q\" (UID: \"999cefec-d3db-47b2-9e0f-05cce59d30d5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.212646 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.212554 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:35.355795 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.355768 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q"] Jul 09 18:56:35.357275 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:56:35.357244 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999cefec_d3db_47b2_9e0f_05cce59d30d5.slice/crio-5011554128e542ea2ababe3b41097df2da51d6f9e310b47040f60d68aabfd217 WatchSource:0}: Error finding container 5011554128e542ea2ababe3b41097df2da51d6f9e310b47040f60d68aabfd217: Status 404 returned error can't find the container with id 5011554128e542ea2ababe3b41097df2da51d6f9e310b47040f60d68aabfd217 Jul 09 18:56:35.358653 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.358637 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:56:35.552785 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:35.552760 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" event={"ID":"999cefec-d3db-47b2-9e0f-05cce59d30d5","Type":"ContainerStarted","Data":"5011554128e542ea2ababe3b41097df2da51d6f9e310b47040f60d68aabfd217"} Jul 09 18:56:36.559519 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:36.559474 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" event={"ID":"999cefec-d3db-47b2-9e0f-05cce59d30d5","Type":"ContainerStarted","Data":"cd1a7d9f7d11398ca1e27a26f85fa6195a2719c743c8b5cc9be0f2abc8ec84ca"} Jul 09 18:56:36.593828 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:36.593768 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" podStartSLOduration=2.412232698 podStartE2EDuration="2.593751805s" totalImagesPullingTime="181.519107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:56:36.589822213 +0000 UTC m=+683.090781251" watchObservedRunningTime="2026-07-09 18:56:36.593751805 +0000 UTC m=+683.094710843" Jul 09 18:56:38.720576 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:38.720544 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:38.721072 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:38.720613 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:38.733614 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:38.733591 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:39.586051 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:39.586023 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wkgp4" Jul 09 18:56:41.390263 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:41.390226 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:56:45.212962 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:45.212912 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:45.213452 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:45.212988 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:45.225487 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:45.225468 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:45.613822 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:45.613730 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-6m25q" Jul 09 18:56:47.834928 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:47.834890 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:56:56.420528 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:56:56.420484 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:57:24.910665 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:57:24.910629 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:57:31.761724 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:57:31.761691 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:57:52.459638 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:57:52.459586 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:02.549108 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:02.549075 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:02.660379 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:02.660346 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:03.743745 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:03.743708 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:03.847126 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:03.847086 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:58:03.849237 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:03.849207 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67bc49c98-8m28h"] Jul 09 18:58:03.849474 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:03.849448 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-67bc49c98-8m28h" podUID="f7811595-763e-42d5-a766-2438b32eb639" containerName="authorino" containerID="cri-o://c751bae94f43c2896cf6c8aa58436d8be45cf00f7ba73bebdfd44806193606f7" gracePeriod=30 Jul 09 18:58:03.999347 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:03.999263 2587 generic.go:356] "Generic (PLEG): container finished" podID="f7811595-763e-42d5-a766-2438b32eb639" containerID="c751bae94f43c2896cf6c8aa58436d8be45cf00f7ba73bebdfd44806193606f7" exitCode=0 Jul 09 18:58:03.999486 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:03.999337 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bc49c98-8m28h" event={"ID":"f7811595-763e-42d5-a766-2438b32eb639","Type":"ContainerDied","Data":"c751bae94f43c2896cf6c8aa58436d8be45cf00f7ba73bebdfd44806193606f7"} Jul 09 18:58:04.106674 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:04.106650 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:58:04.182242 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:04.182211 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7811595-763e-42d5-a766-2438b32eb639-tls-cert\") pod \"f7811595-763e-42d5-a766-2438b32eb639\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " Jul 09 18:58:04.182405 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:04.182264 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f2vz\" (UniqueName: \"kubernetes.io/projected/f7811595-763e-42d5-a766-2438b32eb639-kube-api-access-6f2vz\") pod \"f7811595-763e-42d5-a766-2438b32eb639\" (UID: \"f7811595-763e-42d5-a766-2438b32eb639\") " Jul 09 18:58:04.184207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:04.184171 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7811595-763e-42d5-a766-2438b32eb639-kube-api-access-6f2vz" (OuterVolumeSpecName: "kube-api-access-6f2vz") pod "f7811595-763e-42d5-a766-2438b32eb639" (UID: "f7811595-763e-42d5-a766-2438b32eb639"). InnerVolumeSpecName "kube-api-access-6f2vz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:58:04.199997 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:04.199970 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7811595-763e-42d5-a766-2438b32eb639-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f7811595-763e-42d5-a766-2438b32eb639" (UID: "f7811595-763e-42d5-a766-2438b32eb639"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:58:04.283236 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:04.283161 2587 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7811595-763e-42d5-a766-2438b32eb639-tls-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:58:04.283236 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:04.283186 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f2vz\" (UniqueName: \"kubernetes.io/projected/f7811595-763e-42d5-a766-2438b32eb639-kube-api-access-6f2vz\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:58:05.004914 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:05.004868 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67bc49c98-8m28h" event={"ID":"f7811595-763e-42d5-a766-2438b32eb639","Type":"ContainerDied","Data":"8448c270e777458a1a1ce953caf62396186039be50634f3b13ad172f741eb1ca"} Jul 09 18:58:05.004914 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:05.004900 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67bc49c98-8m28h" Jul 09 18:58:05.005512 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:05.004930 2587 scope.go:117] "RemoveContainer" containerID="c751bae94f43c2896cf6c8aa58436d8be45cf00f7ba73bebdfd44806193606f7" Jul 09 18:58:05.031427 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:05.031403 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67bc49c98-8m28h"] Jul 09 18:58:05.035509 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:05.035488 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67bc49c98-8m28h"] Jul 09 18:58:06.066422 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:58:06.066389 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7811595-763e-42d5-a766-2438b32eb639" path="/var/lib/kubelet/pods/f7811595-763e-42d5-a766-2438b32eb639/volumes" Jul 09 18:59:01.229932 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:01.229887 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:14.686840 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:14.686790 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:14.797065 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:14.797036 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-6vmvp"] Jul 09 18:59:14.797351 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:14.797213 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" podUID="6b74a610-4340-4457-86e8-c504f937f95e" containerName="manager" containerID="cri-o://ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044" gracePeriod=30 Jul 09 18:59:15.048278 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.048249 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:59:15.189659 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.189631 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rghz\" (UniqueName: \"kubernetes.io/projected/6b74a610-4340-4457-86e8-c504f937f95e-kube-api-access-9rghz\") pod \"6b74a610-4340-4457-86e8-c504f937f95e\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " Jul 09 18:59:15.189838 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.189687 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b74a610-4340-4457-86e8-c504f937f95e-webhook-cert\") pod \"6b74a610-4340-4457-86e8-c504f937f95e\" (UID: \"6b74a610-4340-4457-86e8-c504f937f95e\") " Jul 09 18:59:15.191652 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.191624 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b74a610-4340-4457-86e8-c504f937f95e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6b74a610-4340-4457-86e8-c504f937f95e" (UID: "6b74a610-4340-4457-86e8-c504f937f95e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:59:15.191855 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.191832 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b74a610-4340-4457-86e8-c504f937f95e-kube-api-access-9rghz" (OuterVolumeSpecName: "kube-api-access-9rghz") pod "6b74a610-4340-4457-86e8-c504f937f95e" (UID: "6b74a610-4340-4457-86e8-c504f937f95e"). InnerVolumeSpecName "kube-api-access-9rghz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:59:15.290580 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.290509 2587 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b74a610-4340-4457-86e8-c504f937f95e-webhook-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:59:15.290580 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.290536 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rghz\" (UniqueName: \"kubernetes.io/projected/6b74a610-4340-4457-86e8-c504f937f95e-kube-api-access-9rghz\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 18:59:15.343566 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.343531 2587 generic.go:356] "Generic (PLEG): container finished" podID="6b74a610-4340-4457-86e8-c504f937f95e" containerID="ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044" exitCode=0 Jul 09 18:59:15.343743 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.343589 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" Jul 09 18:59:15.343743 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.343617 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" event={"ID":"6b74a610-4340-4457-86e8-c504f937f95e","Type":"ContainerDied","Data":"ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044"} Jul 09 18:59:15.343743 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.343664 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7f6c8fdb-6vmvp" event={"ID":"6b74a610-4340-4457-86e8-c504f937f95e","Type":"ContainerDied","Data":"9a673c56434623154ba9cd17983baf964ce8e285c4665f32da2358707cf8541a"} Jul 09 18:59:15.343743 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.343684 2587 scope.go:117] "RemoveContainer" containerID="ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044" Jul 09 18:59:15.352877 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.352861 2587 scope.go:117] "RemoveContainer" containerID="ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044" Jul 09 18:59:15.353144 ip-10-0-134-249 kubenswrapper[2587]: E0709 18:59:15.353129 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044\": container with ID starting with ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044 not found: ID does not exist" containerID="ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044" Jul 09 18:59:15.353207 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.353151 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044"} err="failed to get container status \"ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044\": rpc error: code = NotFound desc = could not find container \"ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044\": container with ID starting with ddf4395536db6e38c17574e5a67bdcda199b238764db7238a0ab026b814fc044 not found: ID does not exist" Jul 09 18:59:15.367725 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.367696 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-6vmvp"] Jul 09 18:59:15.372026 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:15.372005 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-6vmvp"] Jul 09 18:59:16.067584 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.067547 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:16.067584 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.067584 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b74a610-4340-4457-86e8-c504f937f95e" path="/var/lib/kubelet/pods/6b74a610-4340-4457-86e8-c504f937f95e/volumes" Jul 09 18:59:16.200379 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.200347 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-rb9hw"] Jul 09 18:59:16.200715 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.200704 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b74a610-4340-4457-86e8-c504f937f95e" containerName="manager" Jul 09 18:59:16.200759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.200717 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b74a610-4340-4457-86e8-c504f937f95e" containerName="manager" Jul 09 18:59:16.200759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.200743 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7811595-763e-42d5-a766-2438b32eb639" containerName="authorino" Jul 09 18:59:16.200759 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.200748 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7811595-763e-42d5-a766-2438b32eb639" containerName="authorino" Jul 09 18:59:16.200852 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.200817 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b74a610-4340-4457-86e8-c504f937f95e" containerName="manager" Jul 09 18:59:16.200852 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.200828 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7811595-763e-42d5-a766-2438b32eb639" containerName="authorino" Jul 09 18:59:16.204953 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.204927 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.209232 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.209214 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 18:59:16.210679 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.210659 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 18:59:16.210793 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.210687 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-km6s9\"" Jul 09 18:59:16.217810 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.217787 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-rb9hw"] Jul 09 18:59:16.298910 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.298876 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfe97171-2f03-4a94-b8ab-aa32e3d8681d-webhook-cert\") pod \"maas-controller-5b7f6c8fdb-rb9hw\" (UID: \"cfe97171-2f03-4a94-b8ab-aa32e3d8681d\") " pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.299088 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.298923 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxdz8\" (UniqueName: \"kubernetes.io/projected/cfe97171-2f03-4a94-b8ab-aa32e3d8681d-kube-api-access-hxdz8\") pod \"maas-controller-5b7f6c8fdb-rb9hw\" (UID: \"cfe97171-2f03-4a94-b8ab-aa32e3d8681d\") " pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.399983 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.399957 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfe97171-2f03-4a94-b8ab-aa32e3d8681d-webhook-cert\") pod \"maas-controller-5b7f6c8fdb-rb9hw\" (UID: \"cfe97171-2f03-4a94-b8ab-aa32e3d8681d\") " pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.400134 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.399997 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxdz8\" (UniqueName: \"kubernetes.io/projected/cfe97171-2f03-4a94-b8ab-aa32e3d8681d-kube-api-access-hxdz8\") pod \"maas-controller-5b7f6c8fdb-rb9hw\" (UID: \"cfe97171-2f03-4a94-b8ab-aa32e3d8681d\") " pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.402494 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.402474 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfe97171-2f03-4a94-b8ab-aa32e3d8681d-webhook-cert\") pod \"maas-controller-5b7f6c8fdb-rb9hw\" (UID: \"cfe97171-2f03-4a94-b8ab-aa32e3d8681d\") " pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.412927 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.412896 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxdz8\" (UniqueName: \"kubernetes.io/projected/cfe97171-2f03-4a94-b8ab-aa32e3d8681d-kube-api-access-hxdz8\") pod \"maas-controller-5b7f6c8fdb-rb9hw\" (UID: \"cfe97171-2f03-4a94-b8ab-aa32e3d8681d\") " pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.514980 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.514927 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:16.657858 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:16.657824 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b7f6c8fdb-rb9hw"] Jul 09 18:59:16.660179 ip-10-0-134-249 kubenswrapper[2587]: W0709 18:59:16.660154 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe97171_2f03_4a94_b8ab_aa32e3d8681d.slice/crio-22768a4b3e8acaf53f368ca09214b96f51f0e225bae0d6d8391e5bf9f1b60657 WatchSource:0}: Error finding container 22768a4b3e8acaf53f368ca09214b96f51f0e225bae0d6d8391e5bf9f1b60657: Status 404 returned error can't find the container with id 22768a4b3e8acaf53f368ca09214b96f51f0e225bae0d6d8391e5bf9f1b60657 Jul 09 18:59:17.356819 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:17.356785 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" event={"ID":"cfe97171-2f03-4a94-b8ab-aa32e3d8681d","Type":"ContainerStarted","Data":"d13c55e86e2ed4bf1c024d0db27290d91ba6ba246b2e7096423b204f45c4b28b"} Jul 09 18:59:17.357241 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:17.356836 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" event={"ID":"cfe97171-2f03-4a94-b8ab-aa32e3d8681d","Type":"ContainerStarted","Data":"22768a4b3e8acaf53f368ca09214b96f51f0e225bae0d6d8391e5bf9f1b60657"} Jul 09 18:59:17.357241 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:17.356929 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:17.399387 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:17.399342 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" podStartSLOduration=0.925815743 podStartE2EDuration="1.399328944s" totalImagesPullingTime="473.513201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:17.397137288 +0000 UTC m=+843.898096337" watchObservedRunningTime="2026-07-09 18:59:17.399328944 +0000 UTC m=+843.900287981" Jul 09 18:59:22.567171 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:22.567112 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:22.634502 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:22.634465 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:28.368778 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:28.368744 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b7f6c8fdb-rb9hw" Jul 09 18:59:37.928291 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:37.928253 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:38.084349 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:38.084312 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:54.836623 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:54.836586 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:54.936699 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:54.936664 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 18:59:55.059701 ip-10-0-134-249 kubenswrapper[2587]: I0709 18:59:55.059666 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:02.128442 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:02.128409 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:02.191916 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:02.191885 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:02.271315 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:02.271279 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:02.615452 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:02.615416 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:09.979566 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:09.979493 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:10.071814 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:10.071785 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:14.031281 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:14.031252 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:00:14.034448 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:14.034425 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:00:14.035448 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:14.035427 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:00:14.041773 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:14.041751 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:00:50.466091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:50.466030 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:00:55.838713 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:00:55.838667 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:01:15.646730 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:01:15.646696 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:03:48.134779 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:03:48.134751 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:04:04.675399 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:04.675363 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5"] Jul 09 19:04:04.675852 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:04.675542 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" podUID="8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" containerName="manager" containerID="cri-o://2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760" gracePeriod=10 Jul 09 19:04:08.048166 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.048143 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 19:04:08.098319 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.098227 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq8lk\" (UniqueName: \"kubernetes.io/projected/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-kube-api-access-rq8lk\") pod \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " Jul 09 19:04:08.098319 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.098299 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-extensions-socket-volume\") pod \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\" (UID: \"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5\") " Jul 09 19:04:08.098766 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.098734 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" (UID: "8cbb1b2d-2abf-4882-90db-4a41c24a8dc5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:04:08.100402 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.100372 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-kube-api-access-rq8lk" (OuterVolumeSpecName: "kube-api-access-rq8lk") pod "8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" (UID: "8cbb1b2d-2abf-4882-90db-4a41c24a8dc5"). InnerVolumeSpecName "kube-api-access-rq8lk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:04:08.199239 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.199206 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-extensions-socket-volume\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.199239 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.199234 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq8lk\" (UniqueName: \"kubernetes.io/projected/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5-kube-api-access-rq8lk\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:04:08.788687 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.788651 2587 generic.go:356] "Generic (PLEG): container finished" podID="8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" containerID="2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760" exitCode=0 Jul 09 19:04:08.788865 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.788712 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" Jul 09 19:04:08.788865 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.788736 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" event={"ID":"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5","Type":"ContainerDied","Data":"2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760"} Jul 09 19:04:08.788865 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.788837 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5" event={"ID":"8cbb1b2d-2abf-4882-90db-4a41c24a8dc5","Type":"ContainerDied","Data":"95a00395d8660a1960f9fb4be28ec688353fae694c80b184f134ce21c0021833"} Jul 09 19:04:08.788865 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.788864 2587 scope.go:117] "RemoveContainer" containerID="2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760" Jul 09 19:04:08.799037 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.799019 2587 scope.go:117] "RemoveContainer" containerID="2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760" Jul 09 19:04:08.799316 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:04:08.799297 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760\": container with ID starting with 2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760 not found: ID does not exist" containerID="2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760" Jul 09 19:04:08.799370 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.799324 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760"} err="failed to get container status \"2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760\": rpc error: code = NotFound desc = could not find container \"2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760\": container with ID starting with 2edb8e808becde0bd72e26843005b12c0df93558012ceb32ae53bc1792b40760 not found: ID does not exist" Jul 09 19:04:08.812586 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.812561 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5"] Jul 09 19:04:08.817293 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:08.817273 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zlzh5"] Jul 09 19:04:10.065448 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:04:10.065415 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" path="/var/lib/kubelet/pods/8cbb1b2d-2abf-4882-90db-4a41c24a8dc5/volumes" Jul 09 19:05:05.906307 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.906217 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g"] Jul 09 19:05:05.906784 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.906617 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" containerName="manager" Jul 09 19:05:05.906784 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.906628 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" containerName="manager" Jul 09 19:05:05.906784 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.906702 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cbb1b2d-2abf-4882-90db-4a41c24a8dc5" containerName="manager" Jul 09 19:05:05.909487 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.909470 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:05.912710 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.912694 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-btljp\"" Jul 09 19:05:05.923669 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.923648 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g"] Jul 09 19:05:05.995194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.995158 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/975e2533-0ce3-4706-922f-98f13fa74100-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6rk7g\" (UID: \"975e2533-0ce3-4706-922f-98f13fa74100\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:05.995194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:05.995200 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7pcz\" (UniqueName: \"kubernetes.io/projected/975e2533-0ce3-4706-922f-98f13fa74100-kube-api-access-b7pcz\") pod \"kuadrant-operator-controller-manager-86878c769f-6rk7g\" (UID: \"975e2533-0ce3-4706-922f-98f13fa74100\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:06.096226 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:06.096193 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/975e2533-0ce3-4706-922f-98f13fa74100-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6rk7g\" (UID: \"975e2533-0ce3-4706-922f-98f13fa74100\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:06.096226 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:06.096227 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7pcz\" (UniqueName: \"kubernetes.io/projected/975e2533-0ce3-4706-922f-98f13fa74100-kube-api-access-b7pcz\") pod \"kuadrant-operator-controller-manager-86878c769f-6rk7g\" (UID: \"975e2533-0ce3-4706-922f-98f13fa74100\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:06.096567 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:06.096549 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/975e2533-0ce3-4706-922f-98f13fa74100-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6rk7g\" (UID: \"975e2533-0ce3-4706-922f-98f13fa74100\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:06.107354 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:06.107325 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7pcz\" (UniqueName: \"kubernetes.io/projected/975e2533-0ce3-4706-922f-98f13fa74100-kube-api-access-b7pcz\") pod \"kuadrant-operator-controller-manager-86878c769f-6rk7g\" (UID: \"975e2533-0ce3-4706-922f-98f13fa74100\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:06.219884 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:06.219803 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:06.377661 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:06.377622 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g"] Jul 09 19:05:06.379102 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:05:06.379078 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975e2533_0ce3_4706_922f_98f13fa74100.slice/crio-1d1bc4961565ac320f2ba88eaad8d65fd8001690af4c5540b9a7b134f4616188 WatchSource:0}: Error finding container 1d1bc4961565ac320f2ba88eaad8d65fd8001690af4c5540b9a7b134f4616188: Status 404 returned error can't find the container with id 1d1bc4961565ac320f2ba88eaad8d65fd8001690af4c5540b9a7b134f4616188 Jul 09 19:05:07.076022 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:07.075985 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" event={"ID":"975e2533-0ce3-4706-922f-98f13fa74100","Type":"ContainerStarted","Data":"9ef8833847cbc43f1ea985899b108488d256ec48d59d8680c45e46204176faf6"} Jul 09 19:05:07.076421 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:07.076040 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" event={"ID":"975e2533-0ce3-4706-922f-98f13fa74100","Type":"ContainerStarted","Data":"1d1bc4961565ac320f2ba88eaad8d65fd8001690af4c5540b9a7b134f4616188"} Jul 09 19:05:07.076421 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:07.076080 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:05:07.102650 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:07.102554 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" podStartSLOduration=2.102536784 podStartE2EDuration="2.102536784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:05:07.10152842 +0000 UTC m=+1193.602487480" watchObservedRunningTime="2026-07-09 19:05:07.102536784 +0000 UTC m=+1193.603495823" Jul 09 19:05:14.064522 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:14.064484 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:05:14.069064 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:14.069042 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:05:14.071331 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:14.071305 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:05:14.075562 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:14.075541 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:05:18.082258 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:05:18.082231 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6rk7g" Jul 09 19:06:04.125006 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:06:04.124971 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:06:05.329173 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:06:05.329136 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:07:44.741157 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:07:44.741066 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:06.358428 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:08:06.358385 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:08:06.428485 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:08:06.428459 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:10:14.094893 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:10:14.094867 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:10:14.098850 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:10:14.098826 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:10:14.105484 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:10:14.105455 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:10:14.110191 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:10:14.110171 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:10:53.159409 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:10:53.159321 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:00.024895 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:00.024844 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:08.274355 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:08.274310 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:13.374219 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:13.374178 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:16.262426 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:16.262386 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:16.364493 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:16.364463 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:16.764010 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:16.763980 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:23.152125 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.152090 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:23.252046 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.252010 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:23.328866 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.328838 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj"] Jul 09 19:11:23.332359 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.332335 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.335876 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.335851 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-751b3f0c-openshift-default-dockercfg-ksvn7\"" Jul 09 19:11:23.355555 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.355532 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj"] Jul 09 19:11:23.363015 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.362987 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:23.440524 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440448 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-credential-socket\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440524 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440492 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-podinfo\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440710 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440560 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-token\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440750 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440721 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fd16f1-2096-4a63-8558-03a9c9e8e902-istiod-ca-cert\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440784 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440760 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-certs\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440819 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440789 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-socket\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440854 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440838 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-data\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440883 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hthk8\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-kube-api-access-hthk8\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.440935 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.440918 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-envoy\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.462839 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.462809 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:23.542389 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542361 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fd16f1-2096-4a63-8558-03a9c9e8e902-istiod-ca-cert\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542389 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542390 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-certs\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542416 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-socket\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542456 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-data\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542482 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hthk8\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-kube-api-access-hthk8\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542521 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-envoy\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542543 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-credential-socket\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542565 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-podinfo\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542595 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-token\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542965 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542822 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-certs\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.542965 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542852 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-data\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.543060 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.542937 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-socket\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.543108 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.543087 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-credential-socket\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.543235 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.543221 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fd16f1-2096-4a63-8558-03a9c9e8e902-istiod-ca-cert\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.544702 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.544683 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-envoy\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.545061 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.545046 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-podinfo\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.555479 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.555460 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-token\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.555635 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.555617 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hthk8\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-kube-api-access-hthk8\") pod \"e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.644635 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.644607 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:23.798879 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.798846 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj"] Jul 09 19:11:23.802166 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:11:23.802138 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fd16f1_2096_4a63_8558_03a9c9e8e902.slice/crio-7bab5ec79326a164e1e65f463b69c909a8824938f2b7ffa0c27ed2dd432127f0 WatchSource:0}: Error finding container 7bab5ec79326a164e1e65f463b69c909a8824938f2b7ffa0c27ed2dd432127f0: Status 404 returned error can't find the container with id 7bab5ec79326a164e1e65f463b69c909a8824938f2b7ffa0c27ed2dd432127f0 Jul 09 19:11:23.804205 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.804175 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:23.804298 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.804248 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:23.804364 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.804316 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:23.905641 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.905604 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" event={"ID":"56fd16f1-2096-4a63-8558-03a9c9e8e902","Type":"ContainerStarted","Data":"804a9e4dca5d308bd3b6dc65d9f4cef19f1ec38c1dbfdb4cb04dcfc993f9c736"} Jul 09 19:11:23.905791 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.905674 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" event={"ID":"56fd16f1-2096-4a63-8558-03a9c9e8e902","Type":"ContainerStarted","Data":"7bab5ec79326a164e1e65f463b69c909a8824938f2b7ffa0c27ed2dd432127f0"} Jul 09 19:11:23.929865 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:23.929817 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" podStartSLOduration=0.929803555 podStartE2EDuration="929.803555ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:23.927828089 +0000 UTC m=+1570.428787138" watchObservedRunningTime="2026-07-09 19:11:23.929803555 +0000 UTC m=+1570.430762593" Jul 09 19:11:24.541548 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.541512 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:24.631847 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.631754 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:24.645565 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.645534 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:24.649935 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.649911 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:24.699619 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.699594 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:24.859831 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.859751 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:24.910708 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.910671 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:24.911524 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:24.911506 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:25.129463 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:25.129382 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:25.229693 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:25.229638 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:27.576863 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:27.576830 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:29.640594 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:29.640503 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:29.782927 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:29.782886 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj"] Jul 09 19:11:29.783166 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:29.783137 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" podUID="56fd16f1-2096-4a63-8558-03a9c9e8e902" containerName="istio-proxy" containerID="cri-o://804a9e4dca5d308bd3b6dc65d9f4cef19f1ec38c1dbfdb4cb04dcfc993f9c736" gracePeriod=30 Jul 09 19:11:29.808981 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:29.808933 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:29.908240 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:29.908210 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:30.148968 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:30.143593 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:31.611646 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:31.611612 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:36.001184 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.001161 2587 generic.go:356] "Generic (PLEG): container finished" podID="56fd16f1-2096-4a63-8558-03a9c9e8e902" containerID="804a9e4dca5d308bd3b6dc65d9f4cef19f1ec38c1dbfdb4cb04dcfc993f9c736" exitCode=0 Jul 09 19:11:36.001479 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.001232 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" event={"ID":"56fd16f1-2096-4a63-8558-03a9c9e8e902","Type":"ContainerDied","Data":"804a9e4dca5d308bd3b6dc65d9f4cef19f1ec38c1dbfdb4cb04dcfc993f9c736"} Jul 09 19:11:36.136213 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.136186 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:36.178369 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.178340 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:36.245257 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.245174 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:36.258201 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258176 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-podinfo\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258333 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258233 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-envoy\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258333 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258264 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fd16f1-2096-4a63-8558-03a9c9e8e902-istiod-ca-cert\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258333 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258286 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-credential-socket\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258333 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258324 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hthk8\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-kube-api-access-hthk8\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258359 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-data\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258389 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-socket\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258486 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-token\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258518 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-certs\") pod \"56fd16f1-2096-4a63-8558-03a9c9e8e902\" (UID: \"56fd16f1-2096-4a63-8558-03a9c9e8e902\") " Jul 09 19:11:36.258768 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258606 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:36.258824 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258763 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:36.258824 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258776 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56fd16f1-2096-4a63-8558-03a9c9e8e902-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:11:36.258824 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258798 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:36.259024 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258863 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fd16f1-2096-4a63-8558-03a9c9e8e902-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.259024 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258880 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.259024 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258896 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.259024 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.258911 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.259336 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.259311 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-data" (OuterVolumeSpecName: "istio-data") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:36.260495 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.260466 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:11:36.260608 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.260541 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:36.260663 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.260621 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-kube-api-access-hthk8" (OuterVolumeSpecName: "kube-api-access-hthk8") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "kube-api-access-hthk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:11:36.260704 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.260674 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-token" (OuterVolumeSpecName: "istio-token") pod "56fd16f1-2096-4a63-8558-03a9c9e8e902" (UID: "56fd16f1-2096-4a63-8558-03a9c9e8e902"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:11:36.359964 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.359918 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hthk8\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-kube-api-access-hthk8\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.360077 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.359968 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.360077 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.359983 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.360077 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.359991 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.360077 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.359998 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fd16f1-2096-4a63-8558-03a9c9e8e902-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:36.385727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.385701 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5"] Jul 09 19:11:36.386438 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.386321 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fd16f1-2096-4a63-8558-03a9c9e8e902" containerName="istio-proxy" Jul 09 19:11:36.386438 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.386342 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fd16f1-2096-4a63-8558-03a9c9e8e902" containerName="istio-proxy" Jul 09 19:11:36.386586 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.386456 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="56fd16f1-2096-4a63-8558-03a9c9e8e902" containerName="istio-proxy" Jul 09 19:11:36.388733 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.388718 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.391387 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.391370 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6a618683-openshift-default-dockercfg-z6zvn\"" Jul 09 19:11:36.404640 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.404616 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5"] Jul 09 19:11:36.418353 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.418326 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:36.513829 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.513736 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:36.561920 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.561888 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-data\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.561963 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb15bd5-ddf3-4e63-858a-b1390450c002-istiod-ca-cert\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.562006 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-credential-socket\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.562026 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-socket\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.562041 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-envoy\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.562059 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-token\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.562149 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-podinfo\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.562181 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97lj5\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-kube-api-access-97lj5\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.562288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.562209 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-certs\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.662921 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.662887 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb15bd5-ddf3-4e63-858a-b1390450c002-istiod-ca-cert\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.662931 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-credential-socket\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.662974 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-socket\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.662996 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-envoy\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663011 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-token\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663045 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-podinfo\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663075 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97lj5\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-kube-api-access-97lj5\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663111 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-certs\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663490 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663145 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-data\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663833 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663687 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-socket\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663833 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663748 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-credential-socket\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.663833 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663789 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-data\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.664084 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.663842 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-certs\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.664147 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.664111 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb15bd5-ddf3-4e63-858a-b1390450c002-istiod-ca-cert\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.665367 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.665346 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-envoy\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.665523 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.665509 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-podinfo\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.672439 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.672421 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97lj5\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-kube-api-access-97lj5\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.672653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.672636 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-token\") pod \"e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.701626 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.701607 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:36.841881 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.841855 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5"] Jul 09 19:11:36.845156 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:11:36.845128 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb15bd5_ddf3_4e63_858a_b1390450c002.slice/crio-53c931e4d8c78c47d040ef0989093845383420a8300b8e1c2d6d3628723e2404 WatchSource:0}: Error finding container 53c931e4d8c78c47d040ef0989093845383420a8300b8e1c2d6d3628723e2404: Status 404 returned error can't find the container with id 53c931e4d8c78c47d040ef0989093845383420a8300b8e1c2d6d3628723e2404 Jul 09 19:11:36.847292 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.847260 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:36.847351 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.847323 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:36.847398 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:36.847354 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:37.010751 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.010711 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" event={"ID":"7bb15bd5-ddf3-4e63-858a-b1390450c002","Type":"ContainerStarted","Data":"c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d"} Jul 09 19:11:37.011248 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.010777 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" event={"ID":"7bb15bd5-ddf3-4e63-858a-b1390450c002","Type":"ContainerStarted","Data":"53c931e4d8c78c47d040ef0989093845383420a8300b8e1c2d6d3628723e2404"} Jul 09 19:11:37.012214 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.012191 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" event={"ID":"56fd16f1-2096-4a63-8558-03a9c9e8e902","Type":"ContainerDied","Data":"7bab5ec79326a164e1e65f463b69c909a8824938f2b7ffa0c27ed2dd432127f0"} Jul 09 19:11:37.012332 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.012221 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj" Jul 09 19:11:37.012332 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.012249 2587 scope.go:117] "RemoveContainer" containerID="804a9e4dca5d308bd3b6dc65d9f4cef19f1ec38c1dbfdb4cb04dcfc993f9c736" Jul 09 19:11:37.034143 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.034101 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" podStartSLOduration=1.034086655 podStartE2EDuration="1.034086655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:37.032800846 +0000 UTC m=+1583.533759884" watchObservedRunningTime="2026-07-09 19:11:37.034086655 +0000 UTC m=+1583.535045692" Jul 09 19:11:37.049238 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.049177 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj"] Jul 09 19:11:37.052642 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.052619 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-751b3f0c-openshift-default-559cb47869-zb2fj"] Jul 09 19:11:37.534642 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.534602 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:37.637800 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.637743 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:37.702244 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.702217 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:37.707520 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.707351 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:37.758219 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:37.758178 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:38.018270 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:38.018236 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:38.019098 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:38.019082 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:38.070739 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:38.070698 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56fd16f1-2096-4a63-8558-03a9c9e8e902" path="/var/lib/kubelet/pods/56fd16f1-2096-4a63-8558-03a9c9e8e902/volumes" Jul 09 19:11:38.093340 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:38.093312 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:38.221403 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:38.221375 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:38.321877 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:38.321798 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:39.962397 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:39.962364 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:42.687293 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:42.687258 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:43.128017 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:43.127921 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:48.790191 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:48.790151 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5"] Jul 09 19:11:48.792744 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:48.790371 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" podUID="7bb15bd5-ddf3-4e63-858a-b1390450c002" containerName="istio-proxy" containerID="cri-o://c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d" gracePeriod=30 Jul 09 19:11:48.812603 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:48.812571 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:48.937691 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:48.937612 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:49.061690 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.061666 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:49.090522 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.090495 2587 generic.go:356] "Generic (PLEG): container finished" podID="7bb15bd5-ddf3-4e63-858a-b1390450c002" containerID="c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d" exitCode=0 Jul 09 19:11:49.090677 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.090557 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" Jul 09 19:11:49.090677 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.090586 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" event={"ID":"7bb15bd5-ddf3-4e63-858a-b1390450c002","Type":"ContainerDied","Data":"c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d"} Jul 09 19:11:49.090677 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.090652 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5" event={"ID":"7bb15bd5-ddf3-4e63-858a-b1390450c002","Type":"ContainerDied","Data":"53c931e4d8c78c47d040ef0989093845383420a8300b8e1c2d6d3628723e2404"} Jul 09 19:11:49.090801 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.090686 2587 scope.go:117] "RemoveContainer" containerID="c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d" Jul 09 19:11:49.101619 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.101602 2587 scope.go:117] "RemoveContainer" containerID="c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d" Jul 09 19:11:49.101898 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:11:49.101881 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d\": container with ID starting with c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d not found: ID does not exist" containerID="c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d" Jul 09 19:11:49.101985 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.101906 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d"} err="failed to get container status \"c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d\": rpc error: code = NotFound desc = could not find container \"c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d\": container with ID starting with c430a0e4aa1b1a9d8c04c727a371131a792fceb2b0f91c593e761be5c55a7b8d not found: ID does not exist" Jul 09 19:11:49.173345 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173314 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-certs\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173472 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173386 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-podinfo\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173472 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173416 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-socket\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173547 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173471 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-token\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173547 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173519 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb15bd5-ddf3-4e63-858a-b1390450c002-istiod-ca-cert\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173550 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97lj5\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-kube-api-access-97lj5\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173617 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-envoy\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173619 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:49.173809 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173652 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-data\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173809 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173674 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-credential-socket\") pod \"7bb15bd5-ddf3-4e63-858a-b1390450c002\" (UID: \"7bb15bd5-ddf3-4e63-858a-b1390450c002\") " Jul 09 19:11:49.173917 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.173798 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:49.174050 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.174025 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:49.174050 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.174030 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb15bd5-ddf3-4e63-858a-b1390450c002-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:11:49.174202 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.174061 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.174202 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.174082 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.174820 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.174791 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-data" (OuterVolumeSpecName: "istio-data") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:49.175703 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.175685 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:11:49.175789 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.175736 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-token" (OuterVolumeSpecName: "istio-token") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:11:49.176112 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.176084 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:11:49.176197 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.176097 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-kube-api-access-97lj5" (OuterVolumeSpecName: "kube-api-access-97lj5") pod "7bb15bd5-ddf3-4e63-858a-b1390450c002" (UID: "7bb15bd5-ddf3-4e63-858a-b1390450c002"). InnerVolumeSpecName "kube-api-access-97lj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:11:49.275387 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.275359 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.275387 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.275384 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.275387 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.275393 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb15bd5-ddf3-4e63-858a-b1390450c002-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.275617 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.275401 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97lj5\" (UniqueName: \"kubernetes.io/projected/7bb15bd5-ddf3-4e63-858a-b1390450c002-kube-api-access-97lj5\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.275617 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.275411 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.275617 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.275419 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.275617 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.275426 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb15bd5-ddf3-4e63-858a-b1390450c002-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:11:49.415307 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.415271 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5"] Jul 09 19:11:49.430997 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:49.430965 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6a618683-openshift-default-5ddb9dd777-q9gz5"] Jul 09 19:11:50.065660 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:50.065627 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb15bd5-ddf3-4e63-858a-b1390450c002" path="/var/lib/kubelet/pods/7bb15bd5-ddf3-4e63-858a-b1390450c002/volumes" Jul 09 19:11:51.356500 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:51.356462 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:55.540518 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.540480 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:55.641648 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.641614 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:55.733039 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.733002 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6"] Jul 09 19:11:55.733603 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.733584 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb15bd5-ddf3-4e63-858a-b1390450c002" containerName="istio-proxy" Jul 09 19:11:55.733652 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.733608 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb15bd5-ddf3-4e63-858a-b1390450c002" containerName="istio-proxy" Jul 09 19:11:55.733748 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.733730 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb15bd5-ddf3-4e63-858a-b1390450c002" containerName="istio-proxy" Jul 09 19:11:55.738707 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.738683 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.742197 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.742177 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-861591de-openshift-default-dockercfg-b7v9n\"" Jul 09 19:11:55.749019 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.748999 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6"] Jul 09 19:11:55.761909 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.761885 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:55.827576 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.827485 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:55.832592 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.832564 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-certs\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.832759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.832619 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-credential-socket\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.832828 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.832752 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-envoy\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.832828 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.832781 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-data\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.832968 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.832839 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2n25\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-kube-api-access-q2n25\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.833034 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.832937 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-podinfo\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.833096 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.833048 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-token\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.833096 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.833085 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istiod-ca-cert\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.833205 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.833113 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-socket\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.933836 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.933802 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-token\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.933836 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.933834 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istiod-ca-cert\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.933850 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-socket\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.933885 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-certs\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.933916 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-credential-socket\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.934001 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-envoy\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.934022 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-data\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.934045 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2n25\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-kube-api-access-q2n25\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.934084 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-podinfo\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934531 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.934387 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-certs\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934660 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.934636 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-data\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.934727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.934671 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istiod-ca-cert\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.935030 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.935002 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-credential-socket\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.935157 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.935062 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-socket\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.936896 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.936871 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-envoy\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.937218 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.937198 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-podinfo\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.943793 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.943775 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2n25\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-kube-api-access-q2n25\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:55.945183 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:55.945162 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-token\") pod \"e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:56.050492 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:56.050455 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:56.195826 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:56.195800 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6"] Jul 09 19:11:56.197468 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:11:56.197439 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb16102c_46ba_4d3a_ad72_4d0b2d15cb43.slice/crio-0548818e58ee4f0d88abe248496f4ab629f4a53637b440e324359a06464e2f4b WatchSource:0}: Error finding container 0548818e58ee4f0d88abe248496f4ab629f4a53637b440e324359a06464e2f4b: Status 404 returned error can't find the container with id 0548818e58ee4f0d88abe248496f4ab629f4a53637b440e324359a06464e2f4b Jul 09 19:11:56.199606 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:56.199570 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:56.199681 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:56.199662 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:56.199741 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:56.199723 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:11:56.974240 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:56.974190 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:57.076653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.076612 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:57.132765 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.132725 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" event={"ID":"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43","Type":"ContainerStarted","Data":"9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f"} Jul 09 19:11:57.132967 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.132780 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" event={"ID":"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43","Type":"ContainerStarted","Data":"0548818e58ee4f0d88abe248496f4ab629f4a53637b440e324359a06464e2f4b"} Jul 09 19:11:57.159544 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.159494 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" podStartSLOduration=2.159479758 podStartE2EDuration="2.159479758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:57.158275513 +0000 UTC m=+1603.659234575" watchObservedRunningTime="2026-07-09 19:11:57.159479758 +0000 UTC m=+1603.660438795" Jul 09 19:11:57.236937 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.236848 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:57.612073 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.611984 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:57.768064 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.768022 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:57.869222 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:57.869126 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:11:58.051214 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:58.051180 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:58.056671 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:58.056644 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:58.137766 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:58.137683 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:58.138724 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:58.138705 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:11:58.942727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:11:58.942692 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:01.492449 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:01.492412 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6"] Jul 09 19:12:01.492898 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:01.492574 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" podUID="eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" containerName="istio-proxy" containerID="cri-o://9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f" gracePeriod=30 Jul 09 19:12:01.882049 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:01.881932 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:01.982911 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:01.982880 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:02.331315 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:02.331283 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:04.856553 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:04.856510 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:06.757546 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.757487 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:12:06.831469 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831438 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-certs\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831612 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831489 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-data\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831612 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831523 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-socket\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831612 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831552 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-podinfo\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831612 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831608 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2n25\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-kube-api-access-q2n25\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831853 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831634 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-credential-socket\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831853 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831661 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-envoy\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831853 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831712 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istiod-ca-cert\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831853 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831749 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-token\") pod \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\" (UID: \"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43\") " Jul 09 19:12:06.831853 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831725 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:12:06.831853 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831843 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:12:06.832179 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.831935 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:12:06.832179 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.832105 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.832179 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.832123 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.832179 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.832136 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.832503 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.832461 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:12:06.832981 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.832937 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:12:06.834136 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.834112 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:12:06.834369 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.834345 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:12:06.834458 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.834406 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:12:06.834564 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.834538 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-kube-api-access-q2n25" (OuterVolumeSpecName: "kube-api-access-q2n25") pod "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" (UID: "eb16102c-46ba-4d3a-ad72-4d0b2d15cb43"). InnerVolumeSpecName "kube-api-access-q2n25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:12:06.932568 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.932540 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.932568 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.932563 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.932568 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.932571 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.932755 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.932579 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.932755 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.932587 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2n25\" (UniqueName: \"kubernetes.io/projected/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-kube-api-access-q2n25\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:06.932755 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:06.932595 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:12:07.184316 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.184277 2587 generic.go:356] "Generic (PLEG): container finished" podID="eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" containerID="9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f" exitCode=0 Jul 09 19:12:07.184515 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.184324 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" Jul 09 19:12:07.184515 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.184370 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" event={"ID":"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43","Type":"ContainerDied","Data":"9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f"} Jul 09 19:12:07.184515 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.184434 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6" event={"ID":"eb16102c-46ba-4d3a-ad72-4d0b2d15cb43","Type":"ContainerDied","Data":"0548818e58ee4f0d88abe248496f4ab629f4a53637b440e324359a06464e2f4b"} Jul 09 19:12:07.184515 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.184465 2587 scope.go:117] "RemoveContainer" containerID="9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f" Jul 09 19:12:07.194101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.194087 2587 scope.go:117] "RemoveContainer" containerID="9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f" Jul 09 19:12:07.194357 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:12:07.194340 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f\": container with ID starting with 9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f not found: ID does not exist" containerID="9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f" Jul 09 19:12:07.194403 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.194364 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f"} err="failed to get container status \"9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f\": rpc error: code = NotFound desc = could not find container \"9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f\": container with ID starting with 9929e683cdb1d69529fb0423096a165d328397484aef361bc503a2b14432376f not found: ID does not exist" Jul 09 19:12:07.207662 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.207636 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6"] Jul 09 19:12:07.212835 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.212816 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-861591de-openshift-default-74c45bb77f-zj9t6"] Jul 09 19:12:07.842902 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.842874 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:07.918759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:07.918723 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:08.065660 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:08.065630 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" path="/var/lib/kubelet/pods/eb16102c-46ba-4d3a-ad72-4d0b2d15cb43/volumes" Jul 09 19:12:08.331289 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:08.331250 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:08.430614 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:08.430580 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:09.909310 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:09.909278 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:16.802755 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:16.802712 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:16.883733 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:16.883686 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:16.965443 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:16.965407 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:17.073487 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:17.073395 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:17.543958 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:17.543909 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:29.780524 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:29.780485 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:40.420473 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:40.420443 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:43.586518 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:43.586478 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:50.604636 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:50.604600 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:50.744215 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:50.744184 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:12:51.211354 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:12:51.211305 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:06.744099 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:06.744063 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:13.429539 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:13.429501 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:13.533410 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:13.533370 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:13.633484 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:13.633448 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:23.356502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:23.356466 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:24.203636 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:24.203600 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:25.016061 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:25.016027 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:31.782453 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:31.782413 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:31.890931 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:31.890898 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:31.986250 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:31.986209 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:32.114367 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:32.114283 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:32.854877 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:32.854841 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:32.978557 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:32.978529 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:33.076205 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:33.076165 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:34.786791 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:34.786753 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:41.251553 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:41.251469 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:41.351924 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:41.351892 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:41.944727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:41.944692 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:48.562933 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:48.562894 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:48.665720 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:48.665688 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:48.814927 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:48.814843 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:49.115570 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.115497 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl"] Jul 09 19:13:49.115893 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.115881 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" containerName="istio-proxy" Jul 09 19:13:49.115935 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.115894 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" containerName="istio-proxy" Jul 09 19:13:49.116003 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.115972 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb16102c-46ba-4d3a-ad72-4d0b2d15cb43" containerName="istio-proxy" Jul 09 19:13:49.118871 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.118856 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.122418 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.122399 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5b2f785e-openshift-default-dockercfg-t9w8z\"" Jul 09 19:13:49.146935 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.146903 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:49.151004 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.150981 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl"] Jul 09 19:13:49.205520 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205488 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-socket\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205520 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205518 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1724a936-b2be-4fa0-b05f-521661c63200-istio-podinfo\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205553 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-istio-token\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205567 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1724a936-b2be-4fa0-b05f-521661c63200-istiod-ca-cert\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205642 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-data\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205685 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-certs\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205709 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-credential-socket\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205725 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsdf7\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-kube-api-access-zsdf7\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.205923 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.205742 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-envoy\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.280927 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.280889 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:49.307116 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307086 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-istio-token\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307268 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307118 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1724a936-b2be-4fa0-b05f-521661c63200-istiod-ca-cert\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307268 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307146 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-data\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307268 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307182 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-certs\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307268 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307214 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-credential-socket\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307268 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307237 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsdf7\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-kube-api-access-zsdf7\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307268 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307260 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-envoy\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307579 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307366 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-socket\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307579 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307408 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1724a936-b2be-4fa0-b05f-521661c63200-istio-podinfo\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307686 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307637 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-data\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307747 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307690 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-certs\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307747 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307729 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-credential-socket\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307867 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307844 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1724a936-b2be-4fa0-b05f-521661c63200-istiod-ca-cert\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.307924 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.307858 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-socket\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.309676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.309658 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-envoy\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.309922 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.309902 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1724a936-b2be-4fa0-b05f-521661c63200-istio-podinfo\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.318645 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.318622 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-istio-token\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.318820 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.318801 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsdf7\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-kube-api-access-zsdf7\") pod \"e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.429197 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.429163 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:49.586642 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.586619 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl"] Jul 09 19:13:49.588246 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:13:49.588222 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1724a936_b2be_4fa0_b05f_521661c63200.slice/crio-8e0101d261c059b014d1a29a890d252c18bd05e35260bb4f5ef0f150bad00db1 WatchSource:0}: Error finding container 8e0101d261c059b014d1a29a890d252c18bd05e35260bb4f5ef0f150bad00db1: Status 404 returned error can't find the container with id 8e0101d261c059b014d1a29a890d252c18bd05e35260bb4f5ef0f150bad00db1 Jul 09 19:13:49.590474 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.590442 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:13:49.590582 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.590521 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:13:49.590582 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.590566 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:13:49.677123 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.677091 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" event={"ID":"1724a936-b2be-4fa0-b05f-521661c63200","Type":"ContainerStarted","Data":"27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947"} Jul 09 19:13:49.677293 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.677142 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" event={"ID":"1724a936-b2be-4fa0-b05f-521661c63200","Type":"ContainerStarted","Data":"8e0101d261c059b014d1a29a890d252c18bd05e35260bb4f5ef0f150bad00db1"} Jul 09 19:13:49.700711 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:49.700612 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" podStartSLOduration=0.70059288 podStartE2EDuration="700.59288ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:13:49.698308703 +0000 UTC m=+1716.199267762" watchObservedRunningTime="2026-07-09 19:13:49.70059288 +0000 UTC m=+1716.201551918" Jul 09 19:13:50.429828 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:50.429794 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:50.434866 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:50.434843 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:50.682306 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:50.682213 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:50.683167 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:50.683151 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:13:51.914533 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:51.914498 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:52.005863 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:52.005827 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:52.142389 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:52.142353 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:52.654770 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:52.654734 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:52.753048 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:52.753013 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:53.006281 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:53.006178 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:56.417793 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:56.417759 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:57.087180 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:57.087138 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:57.190185 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:57.190147 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:13:57.551039 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:13:57.551007 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:04.350775 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.350738 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:04.469346 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.469311 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:04.755144 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.755052 2587 generic.go:356] "Generic (PLEG): container finished" podID="1724a936-b2be-4fa0-b05f-521661c63200" containerID="27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947" exitCode=0 Jul 09 19:14:04.755363 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.755136 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" event={"ID":"1724a936-b2be-4fa0-b05f-521661c63200","Type":"ContainerDied","Data":"27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947"} Jul 09 19:14:04.755669 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.755649 2587 scope.go:117] "RemoveContainer" containerID="27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947" Jul 09 19:14:04.757029 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.756990 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:04.757121 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.757082 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:04.757163 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.757129 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:04.812297 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:04.812268 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:05.230748 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:05.230702 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:05.331517 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:05.331476 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:05.682403 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:05.682370 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:14:05.763992 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:05.763954 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" event={"ID":"1724a936-b2be-4fa0-b05f-521661c63200","Type":"ContainerStarted","Data":"3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582"} Jul 09 19:14:06.430053 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:06.430019 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:14:06.430711 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:06.430683 2587 patch_prober.go:28] interesting pod/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 19:14:06.430843 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:06.430761 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 19:14:07.430255 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:07.430218 2587 patch_prober.go:28] interesting pod/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 19:14:07.430632 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:07.430271 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 19:14:08.430299 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:08.430265 2587 patch_prober.go:28] interesting pod/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 19:14:08.430692 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:08.430322 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 19:14:09.430546 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:09.430515 2587 patch_prober.go:28] interesting pod/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 19:14:09.430936 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:09.430567 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 19:14:10.433596 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:10.433562 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:14:10.434028 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:10.433930 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:14:10.434700 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:10.434682 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:14:11.572146 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:11.572091 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:11.673693 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:11.673646 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:11.827981 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:11.827896 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl"] Jul 09 19:14:11.854398 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:11.854366 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:12.085466 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:12.085391 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:12.209159 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:12.209118 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:12.302838 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:12.302804 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:12.419883 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:12.419857 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:12.799689 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:12.799597 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" containerID="cri-o://3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582" gracePeriod=30 Jul 09 19:14:18.050512 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.050485 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:14:18.179913 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.179882 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-credential-socket\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180106 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180012 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1724a936-b2be-4fa0-b05f-521661c63200-istiod-ca-cert\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180106 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180043 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1724a936-b2be-4fa0-b05f-521661c63200-istio-podinfo\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180106 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180078 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-socket\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180257 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180146 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-data\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180257 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180173 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-certs\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180257 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180198 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsdf7\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-kube-api-access-zsdf7\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180257 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180231 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-envoy\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180257 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180196 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:14:18.180502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180258 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-istio-token\") pod \"1724a936-b2be-4fa0-b05f-521661c63200\" (UID: \"1724a936-b2be-4fa0-b05f-521661c63200\") " Jul 09 19:14:18.180502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180457 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:14:18.180502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180477 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:14:18.180675 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180655 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.180734 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180681 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.180734 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180698 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.180734 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.180482 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1724a936-b2be-4fa0-b05f-521661c63200-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:14:18.181213 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.181170 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-data" (OuterVolumeSpecName: "istio-data") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:14:18.182533 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.182498 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:14:18.182638 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.182551 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-istio-token" (OuterVolumeSpecName: "istio-token") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:14:18.182699 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.182664 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1724a936-b2be-4fa0-b05f-521661c63200-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:14:18.182812 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.182796 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-kube-api-access-zsdf7" (OuterVolumeSpecName: "kube-api-access-zsdf7") pod "1724a936-b2be-4fa0-b05f-521661c63200" (UID: "1724a936-b2be-4fa0-b05f-521661c63200"). InnerVolumeSpecName "kube-api-access-zsdf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:14:18.282159 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.282131 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.282159 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.282155 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsdf7\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-kube-api-access-zsdf7\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.282159 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.282166 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1724a936-b2be-4fa0-b05f-521661c63200-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.282370 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.282174 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1724a936-b2be-4fa0-b05f-521661c63200-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.282370 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.282190 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1724a936-b2be-4fa0-b05f-521661c63200-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.282370 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.282199 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1724a936-b2be-4fa0-b05f-521661c63200-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:14:18.832140 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.832111 2587 generic.go:356] "Generic (PLEG): container finished" podID="1724a936-b2be-4fa0-b05f-521661c63200" containerID="3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582" exitCode=0 Jul 09 19:14:18.832354 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.832157 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" Jul 09 19:14:18.832354 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.832193 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" event={"ID":"1724a936-b2be-4fa0-b05f-521661c63200","Type":"ContainerDied","Data":"3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582"} Jul 09 19:14:18.832354 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.832256 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl" event={"ID":"1724a936-b2be-4fa0-b05f-521661c63200","Type":"ContainerDied","Data":"8e0101d261c059b014d1a29a890d252c18bd05e35260bb4f5ef0f150bad00db1"} Jul 09 19:14:18.832354 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.832286 2587 scope.go:117] "RemoveContainer" containerID="3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582" Jul 09 19:14:18.841471 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.841455 2587 scope.go:117] "RemoveContainer" containerID="27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947" Jul 09 19:14:18.849143 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.849127 2587 scope.go:117] "RemoveContainer" containerID="3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582" Jul 09 19:14:18.849388 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:14:18.849369 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582\": container with ID starting with 3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582 not found: ID does not exist" containerID="3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582" Jul 09 19:14:18.849434 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.849400 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582"} err="failed to get container status \"3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582\": rpc error: code = NotFound desc = could not find container \"3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582\": container with ID starting with 3d1e124e242d0116c1d0fa4b69df607f8272348670c4fb408ea77887f61e7582 not found: ID does not exist" Jul 09 19:14:18.849434 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.849421 2587 scope.go:117] "RemoveContainer" containerID="27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947" Jul 09 19:14:18.849652 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:14:18.849634 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947\": container with ID starting with 27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947 not found: ID does not exist" containerID="27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947" Jul 09 19:14:18.849770 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.849653 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947"} err="failed to get container status \"27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947\": rpc error: code = NotFound desc = could not find container \"27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947\": container with ID starting with 27359a22058f0aefe06deb2b687225cfd049325ef386375321b9123c52f96947 not found: ID does not exist" Jul 09 19:14:18.856642 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.856620 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl"] Jul 09 19:14:18.863169 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.863146 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5b2f785e-openshift-default-66467dfcc9-wnhdl"] Jul 09 19:14:18.985745 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:18.985669 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:19.065025 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:19.064990 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:19.233095 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:19.233053 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:19.359037 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:19.359003 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:20.066169 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:20.066140 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1724a936-b2be-4fa0-b05f-521661c63200" path="/var/lib/kubelet/pods/1724a936-b2be-4fa0-b05f-521661c63200/volumes" Jul 09 19:14:26.043197 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:26.043145 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:26.143757 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:26.143729 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:35.105600 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.105564 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:35.424797 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.424767 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc"] Jul 09 19:14:35.425176 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.425163 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" Jul 09 19:14:35.425227 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.425177 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" Jul 09 19:14:35.425263 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.425253 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" Jul 09 19:14:35.425263 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.425261 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" Jul 09 19:14:35.429880 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.429865 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.432543 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.432524 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4cb532e2-openshift-default-dockercfg-svvqj\"" Jul 09 19:14:35.443732 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.443709 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc"] Jul 09 19:14:35.455295 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.455272 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:35.537129 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537103 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-envoy\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537136 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-certs\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537156 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-data\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537179 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-socket\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537226 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tgx9\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-kube-api-access-9tgx9\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537474 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537319 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-token\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537474 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537374 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-podinfo\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537550 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537475 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-credential-socket\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.537550 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.537494 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istiod-ca-cert\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.638729 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638690 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-podinfo\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.638899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638766 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-credential-socket\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.638899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638792 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istiod-ca-cert\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.638899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638833 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-envoy\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.638899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638864 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-certs\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.638899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638888 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-data\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.639196 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638912 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-socket\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.639196 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638934 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tgx9\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-kube-api-access-9tgx9\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.639196 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.638981 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-token\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.639362 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.639239 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-credential-socket\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.639653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.639631 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-socket\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.639818 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.639768 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-data\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.639818 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.639700 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-certs\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.640091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.639925 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istiod-ca-cert\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.641438 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.641412 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-podinfo\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.641539 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.641521 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-envoy\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.648248 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.648228 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-token\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.648510 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.648491 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tgx9\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-kube-api-access-9tgx9\") pod \"e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.664068 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.664046 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:35.740583 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.740505 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:35.883898 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.883874 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc"] Jul 09 19:14:35.885545 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:14:35.885506 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e5c812_2b32_42e0_be78_cf739a8a89f9.slice/crio-bd778c80e16182e5f359b3a67e8986d6c8dc02550f5578af7cf2f3902ff89163 WatchSource:0}: Error finding container bd778c80e16182e5f359b3a67e8986d6c8dc02550f5578af7cf2f3902ff89163: Status 404 returned error can't find the container with id bd778c80e16182e5f359b3a67e8986d6c8dc02550f5578af7cf2f3902ff89163 Jul 09 19:14:35.887623 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.887590 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:35.887717 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.887653 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:35.887717 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.887681 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:35.914510 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:35.914488 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" event={"ID":"d0e5c812-2b32-42e0-be78-cf739a8a89f9","Type":"ContainerStarted","Data":"bd778c80e16182e5f359b3a67e8986d6c8dc02550f5578af7cf2f3902ff89163"} Jul 09 19:14:36.921767 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:36.921733 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" event={"ID":"d0e5c812-2b32-42e0-be78-cf739a8a89f9","Type":"ContainerStarted","Data":"e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2"} Jul 09 19:14:36.945585 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:36.945538 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" podStartSLOduration=1.945522108 podStartE2EDuration="1.945522108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:14:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:14:36.944391274 +0000 UTC m=+1763.445350332" watchObservedRunningTime="2026-07-09 19:14:36.945522108 +0000 UTC m=+1763.446481145" Jul 09 19:14:37.078112 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:37.078080 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:37.178678 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:37.178588 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:37.740641 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:37.740610 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:37.745380 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:37.745358 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:37.927006 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:37.926979 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:37.927967 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:37.927933 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:39.095321 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:39.095286 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:39.196087 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:39.196056 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:39.394573 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:39.394472 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:39.618680 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:39.618647 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:39.935091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:39.935051 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:40.037414 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:40.037381 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:43.633067 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:43.633032 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:14:52.002559 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:52.002479 2587 generic.go:356] "Generic (PLEG): container finished" podID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerID="e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2" exitCode=0 Jul 09 19:14:52.002559 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:52.002549 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" event={"ID":"d0e5c812-2b32-42e0-be78-cf739a8a89f9","Type":"ContainerDied","Data":"e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2"} Jul 09 19:14:52.003021 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:52.002903 2587 scope.go:117] "RemoveContainer" containerID="e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2" Jul 09 19:14:52.003871 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:52.003846 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:52.003931 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:52.003917 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:52.003993 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:52.003964 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:14:52.927350 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:52.927313 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:53.009684 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:53.009650 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" event={"ID":"d0e5c812-2b32-42e0-be78-cf739a8a89f9","Type":"ContainerStarted","Data":"30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874"} Jul 09 19:14:53.741407 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:53.741375 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:53.742720 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:53.742689 2587 patch_prober.go:28] interesting pod/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 19:14:53.742861 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:53.742743 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 19:14:54.741765 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:54.741332 2587 patch_prober.go:28] interesting pod/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 19:14:54.741765 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:54.741403 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 19:14:55.741824 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:55.741784 2587 patch_prober.go:28] interesting pod/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 19:14:55.742348 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:55.741855 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 19:14:56.741225 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:56.741174 2587 patch_prober.go:28] interesting pod/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 19:14:56.741539 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:56.741499 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 19:14:57.745397 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:57.745365 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:57.745844 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:57.745748 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:57.746391 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:57.746373 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:14:58.001990 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:14:58.001873 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:14.127599 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:14.127567 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:15:14.132101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:14.132076 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:15:14.138343 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:14.138322 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:15:14.142710 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:14.142696 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:15:21.598444 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:21.598410 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:21.993892 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:21.993856 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:27.727116 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:27.727079 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:15:32.784782 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:15:32.784740 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:08.894541 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:08.894501 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:10.704018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:10.703980 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:17.391270 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:17.391232 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:23.677225 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:23.677192 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:23.740405 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:23.740366 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:23.951186 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:23.951109 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc"] Jul 09 19:16:23.951362 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:23.951337 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" containerID="cri-o://30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874" gracePeriod=30 Jul 09 19:16:23.961441 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:23.961412 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:24.243602 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:24.243508 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:25.297495 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:25.297452 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:25.398073 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:25.398037 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:25.557115 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:25.557020 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:26.247056 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:26.247020 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:26.344271 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:26.344232 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:26.448586 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:26.448550 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:27.747922 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:27.747885 2587 patch_prober.go:28] interesting pod/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 19:16:27.748326 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:27.748003 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 19:16:27.875209 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:27.875183 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:29.197700 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.197678 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:16:29.256796 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256727 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-data\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.256796 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256761 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-credential-socket\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.256796 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256782 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-certs\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.257101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256800 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-podinfo\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.257101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256841 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tgx9\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-kube-api-access-9tgx9\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.257101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256887 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-socket\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.257101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256953 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-token\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.257101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.256985 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istiod-ca-cert\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.257101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.257032 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-envoy\") pod \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\" (UID: \"d0e5c812-2b32-42e0-be78-cf739a8a89f9\") " Jul 09 19:16:29.257394 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.257073 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:16:29.257394 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.257133 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:16:29.257394 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.257361 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.257394 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.257380 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.257716 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.257694 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:16:29.257794 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.257742 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:16:29.258032 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.258013 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:16:29.259722 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.259697 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:16:29.259977 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.259935 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:16:29.260177 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.260147 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:16:29.260328 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.260240 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-kube-api-access-9tgx9" (OuterVolumeSpecName: "kube-api-access-9tgx9") pod "d0e5c812-2b32-42e0-be78-cf739a8a89f9" (UID: "d0e5c812-2b32-42e0-be78-cf739a8a89f9"). InnerVolumeSpecName "kube-api-access-9tgx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:16:29.358472 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.358446 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.358472 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.358470 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.358645 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.358480 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.358645 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.358488 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.358645 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.358496 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0e5c812-2b32-42e0-be78-cf739a8a89f9-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.358645 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.358503 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tgx9\" (UniqueName: \"kubernetes.io/projected/d0e5c812-2b32-42e0-be78-cf739a8a89f9-kube-api-access-9tgx9\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.358645 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.358514 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0e5c812-2b32-42e0-be78-cf739a8a89f9-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:16:29.485549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.485511 2587 generic.go:356] "Generic (PLEG): container finished" podID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerID="30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874" exitCode=0 Jul 09 19:16:29.485739 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.485569 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" Jul 09 19:16:29.485739 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.485602 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" event={"ID":"d0e5c812-2b32-42e0-be78-cf739a8a89f9","Type":"ContainerDied","Data":"30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874"} Jul 09 19:16:29.485739 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.485696 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc" event={"ID":"d0e5c812-2b32-42e0-be78-cf739a8a89f9","Type":"ContainerDied","Data":"bd778c80e16182e5f359b3a67e8986d6c8dc02550f5578af7cf2f3902ff89163"} Jul 09 19:16:29.485739 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.485728 2587 scope.go:117] "RemoveContainer" containerID="30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874" Jul 09 19:16:29.495325 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.495310 2587 scope.go:117] "RemoveContainer" containerID="e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2" Jul 09 19:16:29.503146 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.503124 2587 scope.go:117] "RemoveContainer" containerID="30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874" Jul 09 19:16:29.503381 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:16:29.503362 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874\": container with ID starting with 30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874 not found: ID does not exist" containerID="30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874" Jul 09 19:16:29.503445 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.503389 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874"} err="failed to get container status \"30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874\": rpc error: code = NotFound desc = could not find container \"30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874\": container with ID starting with 30496a82597d5f03ab2d8d596ffbfa385340661bfcb773462ff884540b6ec874 not found: ID does not exist" Jul 09 19:16:29.503445 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.503407 2587 scope.go:117] "RemoveContainer" containerID="e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2" Jul 09 19:16:29.503681 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:16:29.503663 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2\": container with ID starting with e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2 not found: ID does not exist" containerID="e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2" Jul 09 19:16:29.503748 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.503685 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2"} err="failed to get container status \"e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2\": rpc error: code = NotFound desc = could not find container \"e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2\": container with ID starting with e9cafbda815fdbb3a2909e2d4f17497086ea0d349a49d6400f5eedc4463420a2 not found: ID does not exist" Jul 09 19:16:29.510383 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.510336 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc"] Jul 09 19:16:29.515521 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:29.515501 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4cb532e2-openshift-default-5b588857c5-6tslc"] Jul 09 19:16:30.066971 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:30.066913 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" path="/var/lib/kubelet/pods/d0e5c812-2b32-42e0-be78-cf739a8a89f9/volumes" Jul 09 19:16:34.498509 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:34.498468 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:34.601381 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:34.601348 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:35.203431 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:35.203395 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:41.906104 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:41.906004 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:42.001058 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:42.001025 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:52.554755 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:52.554716 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:16:52.907613 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:16:52.907583 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:13.901061 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:13.901028 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:20.185444 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.185404 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:20.250765 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.250730 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:20.563793 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.563709 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:20.852620 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.852548 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk"] Jul 09 19:17:20.852983 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.852969 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" Jul 09 19:17:20.853032 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.852986 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="1724a936-b2be-4fa0-b05f-521661c63200" containerName="istio-proxy" Jul 09 19:17:20.853032 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.852997 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" Jul 09 19:17:20.853032 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.853002 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" Jul 09 19:17:20.853032 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.853014 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" Jul 09 19:17:20.853032 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.853020 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" Jul 09 19:17:20.853190 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.853082 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" Jul 09 19:17:20.853190 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.853093 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e5c812-2b32-42e0-be78-cf739a8a89f9" containerName="istio-proxy" Jul 09 19:17:20.855082 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.855067 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.858000 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.857974 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2259a7-openshift-default-dockercfg-dw5vc\"" Jul 09 19:17:20.868155 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.868122 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk"] Jul 09 19:17:20.877436 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.877410 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:20.997294 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997261 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-certs\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997463 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997298 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-data\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997463 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997317 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-token\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997463 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997344 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istiod-ca-cert\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997463 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997377 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgpzv\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-kube-api-access-hgpzv\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997463 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997412 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-podinfo\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997463 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997429 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-envoy\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997729 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997477 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-socket\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:20.997729 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:20.997563 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-credential-socket\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.067692 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.067656 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:21.098527 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098487 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-socket\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098695 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098536 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-credential-socket\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098695 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098654 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-certs\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098695 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098692 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-data\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098710 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-token\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098743 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istiod-ca-cert\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098769 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgpzv\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-kube-api-access-hgpzv\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098803 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-podinfo\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.098873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098834 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-envoy\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.099153 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.098984 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-socket\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.099153 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.099031 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-credential-socket\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.099153 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.099090 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-certs\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.099153 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.099115 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-data\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.099427 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.099409 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istiod-ca-cert\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.101012 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.100988 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-envoy\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.101216 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.101201 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-podinfo\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.107842 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.107789 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-token\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.107842 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.107835 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgpzv\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-kube-api-access-hgpzv\") pod \"e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.165478 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.165452 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:21.302507 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.302434 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk"] Jul 09 19:17:21.307752 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:17:21.306030 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a576ea4_9cb2_42e9_a2ab_795a291f4409.slice/crio-7c83646c589a75cdc78e5dbe50ea4f0158137dde0da867c027215add976ace6a WatchSource:0}: Error finding container 7c83646c589a75cdc78e5dbe50ea4f0158137dde0da867c027215add976ace6a: Status 404 returned error can't find the container with id 7c83646c589a75cdc78e5dbe50ea4f0158137dde0da867c027215add976ace6a Jul 09 19:17:21.309667 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.309635 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:17:21.309746 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.309729 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:17:21.309786 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.309767 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:17:21.734053 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.734009 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" event={"ID":"9a576ea4-9cb2-42e9-a2ab-795a291f4409","Type":"ContainerStarted","Data":"ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d"} Jul 09 19:17:21.734268 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.734079 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" event={"ID":"9a576ea4-9cb2-42e9-a2ab-795a291f4409","Type":"ContainerStarted","Data":"7c83646c589a75cdc78e5dbe50ea4f0158137dde0da867c027215add976ace6a"} Jul 09 19:17:21.755526 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:21.755479 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" podStartSLOduration=1.75546492 podStartE2EDuration="1.75546492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:17:21.754556576 +0000 UTC m=+1928.255515660" watchObservedRunningTime="2026-07-09 19:17:21.75546492 +0000 UTC m=+1928.256423959" Jul 09 19:17:22.166375 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:22.166342 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:22.171173 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:22.171151 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:22.739489 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:22.739452 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:22.740613 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:22.740589 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:23.849003 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:23.848967 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:23.974999 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:23.974965 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:24.048863 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:24.048831 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:24.385776 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:24.385739 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:24.507738 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:24.507704 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:24.605743 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:24.605709 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:28.217291 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:28.217258 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:28.563583 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:28.563459 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:28.901454 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:28.901427 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:29.000834 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:29.000804 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:31.480156 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:31.480111 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:31.574915 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:31.574877 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:31.647574 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:31.647545 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:32.025220 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:32.025186 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:32.126311 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:32.126281 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:32.203303 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:32.203272 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:35.995369 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:35.995334 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:38.781119 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:38.781063 2587 patch_prober.go:28] interesting pod/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 19:17:38.781603 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:38.781136 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 19:17:39.430899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:39.430855 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:39.489602 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:39.489559 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:40.829293 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:40.829259 2587 generic.go:356] "Generic (PLEG): container finished" podID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerID="ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d" exitCode=0 Jul 09 19:17:40.829711 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:40.829306 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" event={"ID":"9a576ea4-9cb2-42e9-a2ab-795a291f4409","Type":"ContainerDied","Data":"ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d"} Jul 09 19:17:40.829711 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:40.829701 2587 scope.go:117] "RemoveContainer" containerID="ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d" Jul 09 19:17:40.830709 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:40.830679 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:17:40.830776 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:40.830759 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:17:40.830827 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:40.830811 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:17:41.840306 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:41.839418 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" event={"ID":"9a576ea4-9cb2-42e9-a2ab-795a291f4409","Type":"ContainerStarted","Data":"4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f"} Jul 09 19:17:42.166128 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:42.166091 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:42.166409 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:42.166385 2587 patch_prober.go:28] interesting pod/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 19:17:42.166457 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:42.166441 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 19:17:43.166148 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:43.166103 2587 patch_prober.go:28] interesting pod/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 19:17:43.166539 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:43.166169 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 19:17:43.868972 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:43.868741 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:44.166101 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:44.166062 2587 patch_prober.go:28] interesting pod/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 19:17:44.166331 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:44.166132 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 19:17:45.166601 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:45.166566 2587 patch_prober.go:28] interesting pod/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 19:17:45.167088 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:45.166619 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 19:17:45.933929 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:45.933889 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:46.033813 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:46.033780 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:46.170156 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:46.170120 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:46.170562 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:46.170476 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:46.171163 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:46.171145 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:17:54.160686 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:54.160653 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:54.243473 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:54.243436 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:17:59.428044 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:17:59.428006 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:19.278395 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:19.278359 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:20.296397 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:20.296359 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:20.468994 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:20.468929 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:20.858398 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:20.858358 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:20.958117 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:20.958077 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:21.058767 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:21.058734 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:21.226631 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:21.226590 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:25.939506 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:25.939475 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:34.628142 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:34.628111 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:18:58.553298 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:18:58.553246 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:03.948762 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:03.948727 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:04.224149 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:04.224073 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk"] Jul 09 19:19:04.224297 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:04.224282 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" containerID="cri-o://4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f" gracePeriod=30 Jul 09 19:19:04.244721 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:04.244689 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:04.490817 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:04.490730 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:04.680556 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:04.680505 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:05.097136 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:05.097094 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:05.198816 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:05.198768 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:05.302581 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:05.302550 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:05.532129 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:05.532096 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:09.485742 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.485688 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:19:09.569495 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569463 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-data\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.569647 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569588 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istiod-ca-cert\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.569647 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569619 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-podinfo\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.569647 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569641 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-envoy\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.569829 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569673 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-certs\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.569829 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569697 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgpzv\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-kube-api-access-hgpzv\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.569829 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569741 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-socket\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.570014 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569880 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:09.570014 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569923 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-token\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.570014 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.569998 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-credential-socket\") pod \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\" (UID: \"9a576ea4-9cb2-42e9-a2ab-795a291f4409\") " Jul 09 19:19:09.570176 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570033 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:19:09.570176 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570041 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:09.570293 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570238 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-data" (OuterVolumeSpecName: "istio-data") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:09.570376 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570356 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:09.570451 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570432 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.570492 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570459 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.570492 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570473 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.570492 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.570486 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.571960 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.571927 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:19:09.572060 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.572041 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-kube-api-access-hgpzv" (OuterVolumeSpecName: "kube-api-access-hgpzv") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "kube-api-access-hgpzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:19:09.572315 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.572298 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:09.572386 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.572365 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-token" (OuterVolumeSpecName: "istio-token") pod "9a576ea4-9cb2-42e9-a2ab-795a291f4409" (UID: "9a576ea4-9cb2-42e9-a2ab-795a291f4409"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:19:09.671047 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.671024 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.671047 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.671045 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.671213 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.671055 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgpzv\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-kube-api-access-hgpzv\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.671213 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.671064 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a576ea4-9cb2-42e9-a2ab-795a291f4409-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:09.671213 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:09.671074 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a576ea4-9cb2-42e9-a2ab-795a291f4409-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:10.286266 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.286232 2587 generic.go:356] "Generic (PLEG): container finished" podID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerID="4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f" exitCode=0 Jul 09 19:19:10.286439 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.286286 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" Jul 09 19:19:10.286439 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.286285 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" event={"ID":"9a576ea4-9cb2-42e9-a2ab-795a291f4409","Type":"ContainerDied","Data":"4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f"} Jul 09 19:19:10.286439 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.286411 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk" event={"ID":"9a576ea4-9cb2-42e9-a2ab-795a291f4409","Type":"ContainerDied","Data":"7c83646c589a75cdc78e5dbe50ea4f0158137dde0da867c027215add976ace6a"} Jul 09 19:19:10.286439 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.286439 2587 scope.go:117] "RemoveContainer" containerID="4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f" Jul 09 19:19:10.295369 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.295351 2587 scope.go:117] "RemoveContainer" containerID="ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d" Jul 09 19:19:10.302869 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.302853 2587 scope.go:117] "RemoveContainer" containerID="4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f" Jul 09 19:19:10.303122 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:19:10.303100 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f\": container with ID starting with 4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f not found: ID does not exist" containerID="4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f" Jul 09 19:19:10.303195 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.303134 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f"} err="failed to get container status \"4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f\": rpc error: code = NotFound desc = could not find container \"4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f\": container with ID starting with 4fdbac5e102640c498a10085606dcdfa0924cdaed9aa3cbd5a698fde63dff23f not found: ID does not exist" Jul 09 19:19:10.303195 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.303155 2587 scope.go:117] "RemoveContainer" containerID="ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d" Jul 09 19:19:10.303505 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:19:10.303486 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d\": container with ID starting with ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d not found: ID does not exist" containerID="ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d" Jul 09 19:19:10.303574 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.303510 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d"} err="failed to get container status \"ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d\": rpc error: code = NotFound desc = could not find container \"ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d\": container with ID starting with ee43d877f65c7d896513592a697498e6e53119ce137314c6f0cb5490c4111a6d not found: ID does not exist" Jul 09 19:19:10.307288 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.307268 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk"] Jul 09 19:19:10.313964 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:10.313927 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2259a7-openshift-default-64fbc9464f-lfhlk"] Jul 09 19:19:11.525983 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:11.525926 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:11.626415 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:11.626380 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:11.821608 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:11.821523 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:12.059939 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.059907 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:12.065833 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.065809 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" path="/var/lib/kubelet/pods/9a576ea4-9cb2-42e9-a2ab-795a291f4409/volumes" Jul 09 19:19:12.419994 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.419924 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:12.740116 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.740036 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b"] Jul 09 19:19:12.740549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.740468 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" Jul 09 19:19:12.740549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.740481 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" Jul 09 19:19:12.740549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.740495 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" Jul 09 19:19:12.740549 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.740501 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" Jul 09 19:19:12.740758 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.740574 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" Jul 09 19:19:12.743431 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.743413 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.746191 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.746165 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2b3eb7-openshift-default-dockercfg-rx6bw\"" Jul 09 19:19:12.760865 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.760843 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b"] Jul 09 19:19:12.764136 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.764111 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:12.897500 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897466 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xntgd\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-kube-api-access-xntgd\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897521 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-credential-socket\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897577 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-socket\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897596 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-certs\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897648 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-podinfo\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897893 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897679 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istiod-ca-cert\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897893 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897721 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-token\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897893 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897741 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-data\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.897893 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.897870 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-envoy\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.999591 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999321 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-token\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.999591 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999374 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-data\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.999591 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999453 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-envoy\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.999591 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999494 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xntgd\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-kube-api-access-xntgd\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.999591 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999550 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-credential-socket\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:12.999591 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999581 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-socket\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000057 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999604 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-certs\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000057 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999630 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-podinfo\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000057 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:12.999650 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istiod-ca-cert\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000216 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.000116 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-credential-socket\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000216 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.000164 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-certs\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000325 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.000235 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-socket\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000393 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.000324 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-data\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.000533 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.000512 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istiod-ca-cert\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.001912 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.001895 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-envoy\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.002169 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.002151 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-podinfo\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.009845 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.009825 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xntgd\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-kube-api-access-xntgd\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.009845 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.009831 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-token\") pod \"e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.041409 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.041378 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:13.054276 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.054251 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:13.195150 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.195128 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b"] Jul 09 19:19:13.197457 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:19:13.197427 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d5d423_e6a0_4e55_ba5b_382f79a6a6bb.slice/crio-e63fd47b4eaccaf4b1cddc35e958328f6818b4d3237b4afaca7294001a739670 WatchSource:0}: Error finding container e63fd47b4eaccaf4b1cddc35e958328f6818b4d3237b4afaca7294001a739670: Status 404 returned error can't find the container with id e63fd47b4eaccaf4b1cddc35e958328f6818b4d3237b4afaca7294001a739670 Jul 09 19:19:13.199459 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.199425 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:13.199556 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.199489 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:13.199556 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.199519 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:13.304432 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.304396 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" event={"ID":"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb","Type":"ContainerStarted","Data":"303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423"} Jul 09 19:19:13.304632 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.304462 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" event={"ID":"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb","Type":"ContainerStarted","Data":"e63fd47b4eaccaf4b1cddc35e958328f6818b4d3237b4afaca7294001a739670"} Jul 09 19:19:13.327339 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:13.327293 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" podStartSLOduration=1.327276542 podStartE2EDuration="1.327276542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:19:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:19:13.326311626 +0000 UTC m=+2039.827270689" watchObservedRunningTime="2026-07-09 19:19:13.327276542 +0000 UTC m=+2039.828235649" Jul 09 19:19:14.055286 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:14.055259 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:14.065652 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:14.065629 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:14.309214 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:14.309148 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:14.309966 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:14.309931 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:15.346066 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:15.346015 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:15.450920 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:15.450893 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:15.546402 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:15.546363 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:15.759482 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:15.759447 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:16.262427 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:16.262391 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:16.357758 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:16.357727 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:19.847136 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:19.847101 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:20.338772 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.338733 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:20.627446 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.627355 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52"] Jul 09 19:19:20.628018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.627996 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a576ea4-9cb2-42e9-a2ab-795a291f4409" containerName="istio-proxy" Jul 09 19:19:20.630142 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.630122 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.633004 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.632980 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-dc9769-openshift-default-dockercfg-c8ctb\"" Jul 09 19:19:20.643644 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.643624 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52"] Jul 09 19:19:20.656808 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.656780 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:20.767290 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767257 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-envoy\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767464 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767302 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-token\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767464 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767323 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzhbm\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-kube-api-access-jzhbm\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767464 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767374 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-socket\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767464 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767416 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-podinfo\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767606 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767472 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-credential-socket\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767606 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767497 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-data\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767606 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767547 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istiod-ca-cert\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.767606 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.767571 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-certs\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.868763 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868733 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-envoy\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868772 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-token\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868790 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzhbm\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-kube-api-access-jzhbm\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868807 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-socket\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868824 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-podinfo\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868887 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-credential-socket\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868912 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-data\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.868979 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istiod-ca-cert\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869194 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.869008 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-certs\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869557 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.869291 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-socket\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869557 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.869369 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-credential-socket\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869557 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.869419 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-data\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869557 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.869439 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-certs\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.869875 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.869853 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istiod-ca-cert\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.870962 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.870922 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-envoy\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.871247 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.871228 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-podinfo\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.877493 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.877439 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-token\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.878045 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.878025 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzhbm\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-kube-api-access-jzhbm\") pod \"e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.941059 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.941033 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:20.998999 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:20.998897 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:21.083230 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.083126 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52"] Jul 09 19:19:21.086056 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:19:21.086022 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd63ad2dd_e94c_4bed_8c1a_cf22f38f6732.slice/crio-5948a5cae575cf4a07fef8594f1febdfa1cc6fd69246965cd417618113220f83 WatchSource:0}: Error finding container 5948a5cae575cf4a07fef8594f1febdfa1cc6fd69246965cd417618113220f83: Status 404 returned error can't find the container with id 5948a5cae575cf4a07fef8594f1febdfa1cc6fd69246965cd417618113220f83 Jul 09 19:19:21.088044 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.088010 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:21.088131 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.088075 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:21.088131 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.088103 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:21.344823 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.344790 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" event={"ID":"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732","Type":"ContainerStarted","Data":"ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d"} Jul 09 19:19:21.345020 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.344840 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" event={"ID":"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732","Type":"ContainerStarted","Data":"5948a5cae575cf4a07fef8594f1febdfa1cc6fd69246965cd417618113220f83"} Jul 09 19:19:21.367259 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.367216 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" podStartSLOduration=1.3672000180000001 podStartE2EDuration="1.367200018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:19:21.366038526 +0000 UTC m=+2047.866997591" watchObservedRunningTime="2026-07-09 19:19:21.367200018 +0000 UTC m=+2047.868159112" Jul 09 19:19:21.941657 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:21.941628 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:22.947606 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:22.947575 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:23.355031 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:23.354962 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:23.356196 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:23.356169 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:23.382996 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:23.382931 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:23.482263 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:23.482230 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:23.620495 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:23.620392 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:24.080905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:24.080874 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:24.180307 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:24.180273 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:24.378178 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:24.378086 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:27.816962 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:27.816907 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:30.351117 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:30.351075 2587 patch_prober.go:28] interesting pod/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 19:19:30.351617 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:30.351132 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 19:19:31.398069 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:31.398039 2587 generic.go:356] "Generic (PLEG): container finished" podID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerID="303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423" exitCode=0 Jul 09 19:19:31.398705 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:31.398094 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" event={"ID":"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb","Type":"ContainerDied","Data":"303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423"} Jul 09 19:19:31.398705 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:31.398472 2587 scope.go:117] "RemoveContainer" containerID="303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423" Jul 09 19:19:31.399452 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:31.399422 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:31.399501 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:31.399488 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:31.399573 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:31.399556 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:32.404450 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:32.404412 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" event={"ID":"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb","Type":"ContainerStarted","Data":"0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e"} Jul 09 19:19:33.055177 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:33.055130 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:33.055501 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:33.055180 2587 patch_prober.go:28] interesting pod/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 19:19:33.055727 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:33.055666 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 19:19:34.055897 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:34.055553 2587 patch_prober.go:28] interesting pod/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 19:19:34.055897 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:34.055613 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 19:19:35.055679 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:35.055642 2587 patch_prober.go:28] interesting pod/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 19:19:35.056014 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:35.055982 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 19:19:36.003457 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:36.003419 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:36.055967 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:36.055305 2587 patch_prober.go:28] interesting pod/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 19:19:36.055967 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:36.055366 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 19:19:36.388103 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:36.387987 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:36.559018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:36.558982 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:36.647071 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:36.647033 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:36.876134 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:36.876100 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:37.059607 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:37.059534 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:37.059996 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:37.059961 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:37.060562 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:37.060542 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:38.075402 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:38.075359 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:38.174247 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:38.174209 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:39.353667 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.353633 2587 patch_prober.go:28] interesting pod/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:57208->10.132.0.61:15021: read: connection reset by peer" start-of-body= Jul 09 19:19:39.354104 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.353715 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:57208->10.132.0.61:15021: read: connection reset by peer" Jul 09 19:19:39.441226 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.441197 2587 generic.go:356] "Generic (PLEG): container finished" podID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerID="ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d" exitCode=0 Jul 09 19:19:39.441334 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.441275 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" event={"ID":"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732","Type":"ContainerDied","Data":"ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d"} Jul 09 19:19:39.441707 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.441692 2587 scope.go:117] "RemoveContainer" containerID="ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d" Jul 09 19:19:39.442705 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.442677 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:39.442794 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.442757 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:39.442855 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:39.442808 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:40.457257 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:40.457214 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" event={"ID":"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732","Type":"ContainerStarted","Data":"f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422"} Jul 09 19:19:40.942433 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:40.942402 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:40.942709 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:40.942453 2587 patch_prober.go:28] interesting pod/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 19:19:40.942852 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:40.942831 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 19:19:41.942021 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:41.941982 2587 patch_prober.go:28] interesting pod/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 19:19:41.942536 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:41.942051 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 19:19:42.946971 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:42.944291 2587 patch_prober.go:28] interesting pod/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 19:19:42.946971 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:42.944354 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 19:19:42.958653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:42.958600 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:43.058165 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.058132 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:43.202918 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.202837 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b"] Jul 09 19:19:43.203082 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.203042 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" containerID="cri-o://0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e" gracePeriod=30 Jul 09 19:19:43.224920 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.224870 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:43.450490 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.450382 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:43.653643 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.653612 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:43.942478 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.942374 2587 patch_prober.go:28] interesting pod/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 19:19:43.942478 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:43.942439 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 19:19:44.055690 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:44.055634 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:44.164102 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:44.164061 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:44.261571 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:44.261478 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:44.453452 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:44.453419 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:44.945349 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:44.945319 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:44.945704 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:44.945680 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:44.946292 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:44.946271 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:48.451627 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.451598 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:48.499091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.499008 2587 generic.go:356] "Generic (PLEG): container finished" podID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerID="0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e" exitCode=0 Jul 09 19:19:48.499091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.499062 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" Jul 09 19:19:48.499292 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.499102 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" event={"ID":"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb","Type":"ContainerDied","Data":"0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e"} Jul 09 19:19:48.499292 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.499166 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b" event={"ID":"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb","Type":"ContainerDied","Data":"e63fd47b4eaccaf4b1cddc35e958328f6818b4d3237b4afaca7294001a739670"} Jul 09 19:19:48.499292 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.499200 2587 scope.go:117] "RemoveContainer" containerID="0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e" Jul 09 19:19:48.507875 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.507852 2587 scope.go:117] "RemoveContainer" containerID="303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423" Jul 09 19:19:48.515583 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.515568 2587 scope.go:117] "RemoveContainer" containerID="0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e" Jul 09 19:19:48.515797 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:19:48.515778 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e\": container with ID starting with 0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e not found: ID does not exist" containerID="0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e" Jul 09 19:19:48.515850 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.515806 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e"} err="failed to get container status \"0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e\": rpc error: code = NotFound desc = could not find container \"0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e\": container with ID starting with 0e35887f7fde39b354a0e766095b0200e016d9bd6fab6a166f09707ddbefc17e not found: ID does not exist" Jul 09 19:19:48.515850 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.515821 2587 scope.go:117] "RemoveContainer" containerID="303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423" Jul 09 19:19:48.516120 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:19:48.516105 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423\": container with ID starting with 303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423 not found: ID does not exist" containerID="303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423" Jul 09 19:19:48.516171 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.516127 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423"} err="failed to get container status \"303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423\": rpc error: code = NotFound desc = could not find container \"303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423\": container with ID starting with 303f8d39d1e00656f8f3b18f88c9da413ace7807bf44404199fc1b7792e33423 not found: ID does not exist" Jul 09 19:19:48.534488 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534459 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-socket\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534593 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534498 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istiod-ca-cert\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534593 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534542 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-envoy\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534593 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534582 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-certs\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534758 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534609 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-data\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534758 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534717 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-token\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534758 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534725 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:48.534758 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534755 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-podinfo\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534970 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534790 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-credential-socket\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.534970 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534830 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xntgd\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-kube-api-access-xntgd\") pod \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\" (UID: \"e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb\") " Jul 09 19:19:48.535086 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.534981 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:19:48.535140 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.535119 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:48.535140 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.535131 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.535244 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.535153 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.535335 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.535307 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:48.535643 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.535619 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:48.537049 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.537028 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:48.537661 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.537628 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:19:48.537768 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.537703 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-kube-api-access-xntgd" (OuterVolumeSpecName: "kube-api-access-xntgd") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "kube-api-access-xntgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:19:48.537899 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.537878 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" (UID: "e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:19:48.635837 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.635814 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.635837 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.635837 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.636018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.635846 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.636018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.635857 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xntgd\" (UniqueName: \"kubernetes.io/projected/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-kube-api-access-xntgd\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.636018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.635865 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.636018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.635874 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.636018 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.635882 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:48.824489 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.824462 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b"] Jul 09 19:19:48.827325 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:48.827300 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2b3eb7-openshift-default-99d65fb97-tdr7b"] Jul 09 19:19:50.065519 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:50.065487 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" path="/var/lib/kubelet/pods/e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb/volumes" Jul 09 19:19:50.362768 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:50.362648 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:50.459761 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:50.459726 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:50.616327 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:50.616250 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52"] Jul 09 19:19:50.616481 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:50.616456 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" containerID="cri-o://f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422" gracePeriod=30 Jul 09 19:19:50.637661 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:50.637631 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:50.883662 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:50.883580 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:51.245198 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:51.245164 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:51.568903 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:51.568802 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:51.642421 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:51.642364 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:54.184144 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:54.184101 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:54.267989 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:54.267925 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:54.344072 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:54.344031 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:54.444634 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:54.444536 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:54.781151 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:54.781058 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:54.881254 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:54.881218 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:55.879387 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:55.879334 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:56.010937 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.010908 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-socket\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.010937 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.010938 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-podinfo\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011193 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011001 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-token\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011193 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011035 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-envoy\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011193 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011054 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-credential-socket\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011193 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011075 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-data\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011193 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011111 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzhbm\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-kube-api-access-jzhbm\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011193 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011159 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-certs\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011193 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011180 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istiod-ca-cert\") pod \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\" (UID: \"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732\") " Jul 09 19:19:56.011546 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011227 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:56.011546 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011366 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:56.011546 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011520 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.011546 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011541 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.011762 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011569 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:56.011762 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.011609 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:19:56.012106 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.012072 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-data" (OuterVolumeSpecName: "istio-data") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:56.013305 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.013281 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:19:56.013506 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.013475 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:19:56.013506 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.013466 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-token" (OuterVolumeSpecName: "istio-token") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:19:56.013629 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.013522 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-kube-api-access-jzhbm" (OuterVolumeSpecName: "kube-api-access-jzhbm") pod "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" (UID: "d63ad2dd-e94c-4bed-8c1a-cf22f38f6732"). InnerVolumeSpecName "kube-api-access-jzhbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:19:56.112989 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.112927 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.112989 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.112987 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.113167 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.113002 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.113167 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.113015 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.113167 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.113026 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.113167 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.113033 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.113167 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.113040 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzhbm\" (UniqueName: \"kubernetes.io/projected/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732-kube-api-access-jzhbm\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:19:56.544417 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.544384 2587 generic.go:356] "Generic (PLEG): container finished" podID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerID="f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422" exitCode=0 Jul 09 19:19:56.544590 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.544434 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" Jul 09 19:19:56.544590 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.544471 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" event={"ID":"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732","Type":"ContainerDied","Data":"f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422"} Jul 09 19:19:56.544590 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.544531 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52" event={"ID":"d63ad2dd-e94c-4bed-8c1a-cf22f38f6732","Type":"ContainerDied","Data":"5948a5cae575cf4a07fef8594f1febdfa1cc6fd69246965cd417618113220f83"} Jul 09 19:19:56.544590 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.544561 2587 scope.go:117] "RemoveContainer" containerID="f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422" Jul 09 19:19:56.553114 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.553096 2587 scope.go:117] "RemoveContainer" containerID="ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d" Jul 09 19:19:56.562477 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.562438 2587 scope.go:117] "RemoveContainer" containerID="f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422" Jul 09 19:19:56.563222 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:19:56.563192 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422\": container with ID starting with f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422 not found: ID does not exist" containerID="f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422" Jul 09 19:19:56.563310 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.563231 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422"} err="failed to get container status \"f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422\": rpc error: code = NotFound desc = could not find container \"f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422\": container with ID starting with f381d90271ddc0946f7367315a45eeb119741bb9bb114ee8d15d2491b49be422 not found: ID does not exist" Jul 09 19:19:56.563310 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.563254 2587 scope.go:117] "RemoveContainer" containerID="ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d" Jul 09 19:19:56.563707 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:19:56.563685 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d\": container with ID starting with ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d not found: ID does not exist" containerID="ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d" Jul 09 19:19:56.563805 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.563712 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d"} err="failed to get container status \"ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d\": rpc error: code = NotFound desc = could not find container \"ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d\": container with ID starting with ca5c86a33125858bfc918359b5ad8fdfb6fd6771e4bf306b794d082df0da8f1d not found: ID does not exist" Jul 09 19:19:56.565321 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.565303 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52"] Jul 09 19:19:56.572520 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:56.572500 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-dc9769-openshift-default-69ccb7df5c-6dd52"] Jul 09 19:19:57.385636 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:57.385611 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:58.065232 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:58.065198 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" path="/var/lib/kubelet/pods/d63ad2dd-e94c-4bed-8c1a-cf22f38f6732/volumes" Jul 09 19:19:58.701626 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:58.701590 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:59.067475 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.067379 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:59.349686 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.349611 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s"] Jul 09 19:19:59.350264 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350247 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" Jul 09 19:19:59.350318 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350267 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" Jul 09 19:19:59.350318 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350293 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" Jul 09 19:19:59.350318 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350302 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" Jul 09 19:19:59.350415 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350325 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" Jul 09 19:19:59.350415 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350335 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" Jul 09 19:19:59.350477 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350416 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" Jul 09 19:19:59.350477 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.350430 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" Jul 09 19:19:59.353836 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.353807 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.356530 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.356508 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a966ef-openshift-default-dockercfg-9pln7\"" Jul 09 19:19:59.366769 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.366745 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s"] Jul 09 19:19:59.374973 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.374933 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:59.445517 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445481 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4522dc-7af5-491f-bcee-f8af65b704c6-istiod-ca-cert\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.445517 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445517 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-envoy\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.445759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445539 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-token\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.445759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445596 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-socket\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.445759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445620 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-credential-socket\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.445759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445640 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-certs\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.445759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445702 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-data\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.445759 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445743 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht67m\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-kube-api-access-ht67m\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.446011 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.445806 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-podinfo\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546616 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-socket\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546653 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-credential-socket\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546671 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-certs\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546695 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-data\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546727 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht67m\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-kube-api-access-ht67m\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546809 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-podinfo\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546849 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4522dc-7af5-491f-bcee-f8af65b704c6-istiod-ca-cert\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546874 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-envoy\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.546905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.546898 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-token\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.547297 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.547091 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-socket\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.547297 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.547148 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-certs\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.547297 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.547189 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-credential-socket\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.547297 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.547206 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-data\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.547716 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.547693 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4522dc-7af5-491f-bcee-f8af65b704c6-istiod-ca-cert\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.549361 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.549338 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-envoy\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.549513 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.549490 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-podinfo\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.558626 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.558602 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-token\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.558789 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.558772 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht67m\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-kube-api-access-ht67m\") pod \"e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.664556 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.664527 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:19:59.762434 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.762397 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:19:59.824839 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.824804 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s"] Jul 09 19:19:59.827652 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:19:59.827619 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4522dc_7af5_491f_bcee_f8af65b704c6.slice/crio-6eb4d668acf873f2458806e72b0a019b805eb3912b2a6736458ece55785202af WatchSource:0}: Error finding container 6eb4d668acf873f2458806e72b0a019b805eb3912b2a6736458ece55785202af: Status 404 returned error can't find the container with id 6eb4d668acf873f2458806e72b0a019b805eb3912b2a6736458ece55785202af Jul 09 19:19:59.829638 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.829607 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:59.829699 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.829678 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:19:59.829732 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:19:59.829711 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:20:00.567511 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:00.567480 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" event={"ID":"ba4522dc-7af5-491f-bcee-f8af65b704c6","Type":"ContainerStarted","Data":"9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32"} Jul 09 19:20:00.567690 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:00.567537 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" event={"ID":"ba4522dc-7af5-491f-bcee-f8af65b704c6","Type":"ContainerStarted","Data":"6eb4d668acf873f2458806e72b0a019b805eb3912b2a6736458ece55785202af"} Jul 09 19:20:00.594878 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:00.594825 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" podStartSLOduration=1.5948106549999999 podStartE2EDuration="1.594810655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:20:00.592993229 +0000 UTC m=+2087.093952288" watchObservedRunningTime="2026-07-09 19:20:00.594810655 +0000 UTC m=+2087.095769693" Jul 09 19:20:00.665407 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:00.665378 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:00.670862 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:00.670838 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:01.573016 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:01.572982 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:01.574042 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:01.574023 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:02.567372 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:02.567341 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:02.673002 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:02.672967 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:02.743181 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:02.743142 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:03.172428 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:03.172381 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:03.273905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:03.273864 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:03.715734 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:03.715698 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:07.103562 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:07.103527 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:14.161952 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:14.161925 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:20:14.166762 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:14.166738 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:20:14.170794 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:14.170773 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:20:14.175547 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:14.175487 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:20:16.369405 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:16.369351 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:16.468230 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:16.468189 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:16.840245 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:16.839659 2587 patch_prober.go:28] interesting pod/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:60376->10.132.0.62:15021: read: connection reset by peer" start-of-body= Jul 09 19:20:16.840245 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:16.839721 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:60376->10.132.0.62:15021: read: connection reset by peer" Jul 09 19:20:17.668011 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:17.667967 2587 generic.go:356] "Generic (PLEG): container finished" podID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerID="9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32" exitCode=0 Jul 09 19:20:17.668421 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:17.668027 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" event={"ID":"ba4522dc-7af5-491f-bcee-f8af65b704c6","Type":"ContainerDied","Data":"9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32"} Jul 09 19:20:17.668421 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:17.668408 2587 scope.go:117] "RemoveContainer" containerID="9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32" Jul 09 19:20:17.669402 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:17.669373 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:20:17.669496 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:17.669458 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:20:17.669556 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:17.669504 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 19:20:18.674536 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:18.674502 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" event={"ID":"ba4522dc-7af5-491f-bcee-f8af65b704c6","Type":"ContainerStarted","Data":"7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715"} Jul 09 19:20:19.665206 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:19.665172 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:19.665403 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:19.665383 2587 patch_prober.go:28] interesting pod/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 19:20:19.665449 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:19.665429 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 19:20:20.665316 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:20.665278 2587 patch_prober.go:28] interesting pod/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 19:20:20.665742 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:20.665336 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 19:20:21.665617 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:21.665582 2587 patch_prober.go:28] interesting pod/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 19:20:21.666048 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:21.665644 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 19:20:22.644725 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:22.644684 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:22.664927 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:22.664892 2587 patch_prober.go:28] interesting pod/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 19:20:22.665109 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:22.664977 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 19:20:23.668159 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:23.668128 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:23.668523 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:23.668470 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:23.669192 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:23.669174 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:20:25.087497 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:25.087469 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:20:25.190250 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:20:25.190218 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:21:00.740760 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:21:00.740722 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:21:02.206518 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:21:02.206483 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:23:24.297966 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:23:24.297915 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:23:25.321248 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:23:25.321214 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:23:37.830700 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:23:37.830671 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:05.158184 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:05.158148 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:07.036217 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:07.036142 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:27.672721 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:27.672683 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:38.927912 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:38.927875 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:39.025965 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:39.025903 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:50.313093 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:50.313058 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:51.133770 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:51.133736 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:24:52.007336 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:24:52.007304 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:25:14.192885 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:14.192855 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:25:14.197418 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:14.197395 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:25:14.203723 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:14.203704 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:25:14.207533 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:14.207515 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qpr66_f1b9b361-c850-40cd-af98-cd00992eb6d3/ovn-acl-logging/0.log" Jul 09 19:25:15.726403 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:15.726368 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:25:17.843080 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:17.843023 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:25:17.943203 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:17.943166 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:25:26.258981 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:26.258931 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:25:26.363778 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:26.363741 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:25:39.938023 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:39.937894 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:25:49.577424 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:25:49.577391 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:28.795347 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:28.795303 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:29.720600 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:29.720546 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:26:30.707177 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.707146 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7532-140609016818608"] Jul 09 19:26:30.707590 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.707577 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" Jul 09 19:26:30.707634 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.707593 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" Jul 09 19:26:30.707669 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.707654 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d63ad2dd-e94c-4bed-8c1a-cf22f38f6732" containerName="istio-proxy" Jul 09 19:26:30.707669 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.707665 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3d5d423-e6a0-4e55-ba5b-382f79a6a6bb" containerName="istio-proxy" Jul 09 19:26:30.709890 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.709874 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016818608" Jul 09 19:26:30.712502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.712477 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-m9h5b\"" Jul 09 19:26:30.717756 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.717733 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016818608"] Jul 09 19:26:30.804736 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.804703 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpgrd\" (UniqueName: \"kubernetes.io/projected/9ca126b3-2476-4725-894b-d54e96b5b8bc-kube-api-access-bpgrd\") pod \"test-curl-7532-140609016818608\" (UID: \"9ca126b3-2476-4725-894b-d54e96b5b8bc\") " pod="openshift-ingress/test-curl-7532-140609016818608" Jul 09 19:26:30.905714 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.905684 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpgrd\" (UniqueName: \"kubernetes.io/projected/9ca126b3-2476-4725-894b-d54e96b5b8bc-kube-api-access-bpgrd\") pod \"test-curl-7532-140609016818608\" (UID: \"9ca126b3-2476-4725-894b-d54e96b5b8bc\") " pod="openshift-ingress/test-curl-7532-140609016818608" Jul 09 19:26:30.915975 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:30.915937 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpgrd\" (UniqueName: \"kubernetes.io/projected/9ca126b3-2476-4725-894b-d54e96b5b8bc-kube-api-access-bpgrd\") pod \"test-curl-7532-140609016818608\" (UID: \"9ca126b3-2476-4725-894b-d54e96b5b8bc\") " pod="openshift-ingress/test-curl-7532-140609016818608" Jul 09 19:26:31.020079 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:31.020009 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016818608" Jul 09 19:26:31.160287 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:31.160261 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016818608"] Jul 09 19:26:31.162269 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:26:31.162238 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ca126b3_2476_4725_894b_d54e96b5b8bc.slice/crio-a8b96c95402117ec12f7a25f664a15bf379a303b0ebe66d6fef0db3ee59c9650 WatchSource:0}: Error finding container a8b96c95402117ec12f7a25f664a15bf379a303b0ebe66d6fef0db3ee59c9650: Status 404 returned error can't find the container with id a8b96c95402117ec12f7a25f664a15bf379a303b0ebe66d6fef0db3ee59c9650 Jul 09 19:26:31.163788 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:31.163747 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 19:26:31.521896 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:31.521864 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016818608" event={"ID":"9ca126b3-2476-4725-894b-d54e96b5b8bc","Type":"ContainerStarted","Data":"a8b96c95402117ec12f7a25f664a15bf379a303b0ebe66d6fef0db3ee59c9650"} Jul 09 19:26:32.530696 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:32.530660 2587 generic.go:356] "Generic (PLEG): container finished" podID="9ca126b3-2476-4725-894b-d54e96b5b8bc" containerID="df4b8153cae578461b63cd47f51877a81dd092e57abde00a8139bd69a788e440" exitCode=0 Jul 09 19:26:32.531163 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:32.530752 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016818608" event={"ID":"9ca126b3-2476-4725-894b-d54e96b5b8bc","Type":"ContainerDied","Data":"df4b8153cae578461b63cd47f51877a81dd092e57abde00a8139bd69a788e440"} Jul 09 19:26:33.678716 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.678691 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016818608" Jul 09 19:26:33.704692 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.704667 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7532-140609016818608_9ca126b3-2476-4725-894b-d54e96b5b8bc/test-curl-7532-140609016818608/0.log" Jul 09 19:26:33.725655 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.725629 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpgrd\" (UniqueName: \"kubernetes.io/projected/9ca126b3-2476-4725-894b-d54e96b5b8bc-kube-api-access-bpgrd\") pod \"9ca126b3-2476-4725-894b-d54e96b5b8bc\" (UID: \"9ca126b3-2476-4725-894b-d54e96b5b8bc\") " Jul 09 19:26:33.727517 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.727489 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca126b3-2476-4725-894b-d54e96b5b8bc-kube-api-access-bpgrd" (OuterVolumeSpecName: "kube-api-access-bpgrd") pod "9ca126b3-2476-4725-894b-d54e96b5b8bc" (UID: "9ca126b3-2476-4725-894b-d54e96b5b8bc"). InnerVolumeSpecName "kube-api-access-bpgrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:26:33.735084 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.735061 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7532-140609016818608"] Jul 09 19:26:33.740604 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.740583 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7532-140609016818608"] Jul 09 19:26:33.826901 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.826820 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpgrd\" (UniqueName: \"kubernetes.io/projected/9ca126b3-2476-4725-894b-d54e96b5b8bc-kube-api-access-bpgrd\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:26:33.874730 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.874700 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7532-140609016878640"] Jul 09 19:26:33.875154 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.875141 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca126b3-2476-4725-894b-d54e96b5b8bc" containerName="test-curl-7532-140609016818608" Jul 09 19:26:33.875209 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.875156 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca126b3-2476-4725-894b-d54e96b5b8bc" containerName="test-curl-7532-140609016818608" Jul 09 19:26:33.875245 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.875230 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca126b3-2476-4725-894b-d54e96b5b8bc" containerName="test-curl-7532-140609016818608" Jul 09 19:26:33.877384 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.877369 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:33.887077 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.887056 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016878640"] Jul 09 19:26:33.927502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:33.927470 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49ndd\" (UniqueName: \"kubernetes.io/projected/7c63cbd6-1326-40b3-aeb7-33c5467d0332-kube-api-access-49ndd\") pod \"test-curl-7532-140609016878640\" (UID: \"7c63cbd6-1326-40b3-aeb7-33c5467d0332\") " pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:34.028506 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.028472 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49ndd\" (UniqueName: \"kubernetes.io/projected/7c63cbd6-1326-40b3-aeb7-33c5467d0332-kube-api-access-49ndd\") pod \"test-curl-7532-140609016878640\" (UID: \"7c63cbd6-1326-40b3-aeb7-33c5467d0332\") " pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:34.042680 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.042652 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49ndd\" (UniqueName: \"kubernetes.io/projected/7c63cbd6-1326-40b3-aeb7-33c5467d0332-kube-api-access-49ndd\") pod \"test-curl-7532-140609016878640\" (UID: \"7c63cbd6-1326-40b3-aeb7-33c5467d0332\") " pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:34.066149 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.066112 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca126b3-2476-4725-894b-d54e96b5b8bc" path="/var/lib/kubelet/pods/9ca126b3-2476-4725-894b-d54e96b5b8bc/volumes" Jul 09 19:26:34.187230 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.187196 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:34.320457 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.320431 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016878640"] Jul 09 19:26:34.322321 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:26:34.322285 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c63cbd6_1326_40b3_aeb7_33c5467d0332.slice/crio-57f1bf36bcd1d61120cd912d0aa30bfc6f8f27b8c4b30483b1d1dd3c780fc9f7 WatchSource:0}: Error finding container 57f1bf36bcd1d61120cd912d0aa30bfc6f8f27b8c4b30483b1d1dd3c780fc9f7: Status 404 returned error can't find the container with id 57f1bf36bcd1d61120cd912d0aa30bfc6f8f27b8c4b30483b1d1dd3c780fc9f7 Jul 09 19:26:34.543300 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.543217 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016818608" Jul 09 19:26:34.543300 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.543268 2587 scope.go:117] "RemoveContainer" containerID="df4b8153cae578461b63cd47f51877a81dd092e57abde00a8139bd69a788e440" Jul 09 19:26:34.544637 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:34.544423 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016878640" event={"ID":"7c63cbd6-1326-40b3-aeb7-33c5467d0332","Type":"ContainerStarted","Data":"57f1bf36bcd1d61120cd912d0aa30bfc6f8f27b8c4b30483b1d1dd3c780fc9f7"} Jul 09 19:26:35.551385 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:35.551348 2587 generic.go:356] "Generic (PLEG): container finished" podID="7c63cbd6-1326-40b3-aeb7-33c5467d0332" containerID="1e2d31f54cd79ed259cd693e39acf3143f45de6aa8cc8815fc922c01a6505066" exitCode=0 Jul 09 19:26:35.551817 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:35.551433 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016878640" event={"ID":"7c63cbd6-1326-40b3-aeb7-33c5467d0332","Type":"ContainerDied","Data":"1e2d31f54cd79ed259cd693e39acf3143f45de6aa8cc8815fc922c01a6505066"} Jul 09 19:26:36.701574 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.701549 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:36.724859 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.724836 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7532-140609016878640_7c63cbd6-1326-40b3-aeb7-33c5467d0332/test-curl-7532-140609016878640/0.log" Jul 09 19:26:36.751916 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.751892 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7532-140609016878640"] Jul 09 19:26:36.752201 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.752183 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49ndd\" (UniqueName: \"kubernetes.io/projected/7c63cbd6-1326-40b3-aeb7-33c5467d0332-kube-api-access-49ndd\") pod \"7c63cbd6-1326-40b3-aeb7-33c5467d0332\" (UID: \"7c63cbd6-1326-40b3-aeb7-33c5467d0332\") " Jul 09 19:26:36.754102 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.754081 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c63cbd6-1326-40b3-aeb7-33c5467d0332-kube-api-access-49ndd" (OuterVolumeSpecName: "kube-api-access-49ndd") pod "7c63cbd6-1326-40b3-aeb7-33c5467d0332" (UID: "7c63cbd6-1326-40b3-aeb7-33c5467d0332"). InnerVolumeSpecName "kube-api-access-49ndd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:26:36.757864 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.757842 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7532-140609016878640"] Jul 09 19:26:36.853339 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.853279 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49ndd\" (UniqueName: \"kubernetes.io/projected/7c63cbd6-1326-40b3-aeb7-33c5467d0332-kube-api-access-49ndd\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:26:36.920760 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.920737 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:36.921222 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.921206 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c63cbd6-1326-40b3-aeb7-33c5467d0332" containerName="test-curl-7532-140609016878640" Jul 09 19:26:36.921309 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.921224 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c63cbd6-1326-40b3-aeb7-33c5467d0332" containerName="test-curl-7532-140609016878640" Jul 09 19:26:36.921366 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.921327 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c63cbd6-1326-40b3-aeb7-33c5467d0332" containerName="test-curl-7532-140609016878640" Jul 09 19:26:36.924125 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.924107 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:36.929593 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.929574 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:36.953924 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:36.953902 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x25q6\" (UniqueName: \"kubernetes.io/projected/912b79ce-3f86-4038-a07d-8f9f04d06535-kube-api-access-x25q6\") pod \"test-curl-7532-140609016882064\" (UID: \"912b79ce-3f86-4038-a07d-8f9f04d06535\") " pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:37.054826 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:37.054797 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25q6\" (UniqueName: \"kubernetes.io/projected/912b79ce-3f86-4038-a07d-8f9f04d06535-kube-api-access-x25q6\") pod \"test-curl-7532-140609016882064\" (UID: \"912b79ce-3f86-4038-a07d-8f9f04d06535\") " pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:37.063758 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:37.063732 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x25q6\" (UniqueName: \"kubernetes.io/projected/912b79ce-3f86-4038-a07d-8f9f04d06535-kube-api-access-x25q6\") pod \"test-curl-7532-140609016882064\" (UID: \"912b79ce-3f86-4038-a07d-8f9f04d06535\") " pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:37.235401 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:37.235372 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:37.369643 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:37.369613 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:37.370970 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:26:37.370925 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod912b79ce_3f86_4038_a07d_8f9f04d06535.slice/crio-43382d046ec0e7bf04f969c56f74d6dbba52e6919d10bfeebf1073d4e0bea319 WatchSource:0}: Error finding container 43382d046ec0e7bf04f969c56f74d6dbba52e6919d10bfeebf1073d4e0bea319: Status 404 returned error can't find the container with id 43382d046ec0e7bf04f969c56f74d6dbba52e6919d10bfeebf1073d4e0bea319 Jul 09 19:26:37.569529 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:37.569443 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57f1bf36bcd1d61120cd912d0aa30bfc6f8f27b8c4b30483b1d1dd3c780fc9f7" Jul 09 19:26:37.569529 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:37.569447 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:37.570642 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:37.570617 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016882064" event={"ID":"912b79ce-3f86-4038-a07d-8f9f04d06535","Type":"ContainerStarted","Data":"43382d046ec0e7bf04f969c56f74d6dbba52e6919d10bfeebf1073d4e0bea319"} Jul 09 19:26:37.579827 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:26:37.579798 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7532-140609016878640\" is forbidden: User \"system:node:ip-10-0-134-249.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-134-249.ec2.internal' and this object" podUID="7c63cbd6-1326-40b3-aeb7-33c5467d0332" pod="openshift-ingress/test-curl-7532-140609016878640" Jul 09 19:26:38.067249 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:38.067217 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c63cbd6-1326-40b3-aeb7-33c5467d0332" path="/var/lib/kubelet/pods/7c63cbd6-1326-40b3-aeb7-33c5467d0332/volumes" Jul 09 19:26:38.578240 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:38.578207 2587 generic.go:356] "Generic (PLEG): container finished" podID="912b79ce-3f86-4038-a07d-8f9f04d06535" containerID="cd938f2892f2bb07ff20d025deb56172b2d1073e0f8609cf4fa75a0912a53451" exitCode=0 Jul 09 19:26:38.578448 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:38.578296 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016882064" event={"ID":"912b79ce-3f86-4038-a07d-8f9f04d06535","Type":"ContainerDied","Data":"cd938f2892f2bb07ff20d025deb56172b2d1073e0f8609cf4fa75a0912a53451"} Jul 09 19:26:39.741140 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.741113 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:39.764887 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.764862 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7532-140609016882064_912b79ce-3f86-4038-a07d-8f9f04d06535/test-curl-7532-140609016882064/0.log" Jul 09 19:26:39.778294 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.778263 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x25q6\" (UniqueName: \"kubernetes.io/projected/912b79ce-3f86-4038-a07d-8f9f04d06535-kube-api-access-x25q6\") pod \"912b79ce-3f86-4038-a07d-8f9f04d06535\" (UID: \"912b79ce-3f86-4038-a07d-8f9f04d06535\") " Jul 09 19:26:39.780414 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.780389 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/912b79ce-3f86-4038-a07d-8f9f04d06535-kube-api-access-x25q6" (OuterVolumeSpecName: "kube-api-access-x25q6") pod "912b79ce-3f86-4038-a07d-8f9f04d06535" (UID: "912b79ce-3f86-4038-a07d-8f9f04d06535"). InnerVolumeSpecName "kube-api-access-x25q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:26:39.791315 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.791291 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:39.796286 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.796267 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:39.879988 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.879879 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x25q6\" (UniqueName: \"kubernetes.io/projected/912b79ce-3f86-4038-a07d-8f9f04d06535-kube-api-access-x25q6\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:26:39.921802 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.921777 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:39.922208 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.922194 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="912b79ce-3f86-4038-a07d-8f9f04d06535" containerName="test-curl-7532-140609016882064" Jul 09 19:26:39.922280 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.922209 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="912b79ce-3f86-4038-a07d-8f9f04d06535" containerName="test-curl-7532-140609016882064" Jul 09 19:26:39.922280 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.922272 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="912b79ce-3f86-4038-a07d-8f9f04d06535" containerName="test-curl-7532-140609016882064" Jul 09 19:26:39.925184 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.925170 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:39.932014 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.931986 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:39.980507 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:39.980473 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldj44\" (UniqueName: \"kubernetes.io/projected/a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d-kube-api-access-ldj44\") pod \"test-curl-7532-140609016882064\" (UID: \"a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d\") " pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:40.066360 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.066329 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="912b79ce-3f86-4038-a07d-8f9f04d06535" path="/var/lib/kubelet/pods/912b79ce-3f86-4038-a07d-8f9f04d06535/volumes" Jul 09 19:26:40.081352 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.081327 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldj44\" (UniqueName: \"kubernetes.io/projected/a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d-kube-api-access-ldj44\") pod \"test-curl-7532-140609016882064\" (UID: \"a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d\") " pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:40.090848 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.090830 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldj44\" (UniqueName: \"kubernetes.io/projected/a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d-kube-api-access-ldj44\") pod \"test-curl-7532-140609016882064\" (UID: \"a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d\") " pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:40.235216 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.235190 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:40.370427 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.370403 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:40.372379 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:26:40.372352 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3d5993d_fa7d_4c94_a98d_1ef3f2fcc98d.slice/crio-0cdb411423305fecfbfc3d8d592172be166a4cd6fb50eb764ce3f128e150dd03 WatchSource:0}: Error finding container 0cdb411423305fecfbfc3d8d592172be166a4cd6fb50eb764ce3f128e150dd03: Status 404 returned error can't find the container with id 0cdb411423305fecfbfc3d8d592172be166a4cd6fb50eb764ce3f128e150dd03 Jul 09 19:26:40.588922 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.588845 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:40.588922 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.588896 2587 scope.go:117] "RemoveContainer" containerID="cd938f2892f2bb07ff20d025deb56172b2d1073e0f8609cf4fa75a0912a53451" Jul 09 19:26:40.590195 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:40.590033 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016882064" event={"ID":"a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d","Type":"ContainerStarted","Data":"0cdb411423305fecfbfc3d8d592172be166a4cd6fb50eb764ce3f128e150dd03"} Jul 09 19:26:41.025746 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:26:41.025708 2587 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="curlimages/curl:latest" Jul 09 19:26:41.026143 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:26:41.025752 2587 kuberuntime_image.go:43] "Failed to pull image" err="unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="curlimages/curl:latest" Jul 09 19:26:41.026143 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:26:41.025820 2587 kuberuntime_manager.go:1459] "Unhandled Error" err="container test-curl-7532-140609016882064 start failed in pod test-curl-7532-140609016882064_openshift-ingress(a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 09 19:26:41.026143 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:26:41.025850 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"test-curl-7532-140609016882064\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-ingress/test-curl-7532-140609016882064" podUID="a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" Jul 09 19:26:41.596559 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:26:41.596520 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"test-curl-7532-140609016882064\" with ImagePullBackOff: \"Back-off pulling image \\\"curlimages/curl:latest\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-ingress/test-curl-7532-140609016882064" podUID="a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" Jul 09 19:26:54.662544 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:54.662507 2587 generic.go:356] "Generic (PLEG): container finished" podID="a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" containerID="ac72445363e8a9e2b5d3fc362f9a4cf080f85d8082221be00d0e20bd6a1054eb" exitCode=0 Jul 09 19:26:54.662879 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:54.662579 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016882064" event={"ID":"a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d","Type":"ContainerDied","Data":"ac72445363e8a9e2b5d3fc362f9a4cf080f85d8082221be00d0e20bd6a1054eb"} Jul 09 19:26:55.813736 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.813712 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:55.836799 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.836774 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7532-140609016882064_a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d/test-curl-7532-140609016882064/0.log" Jul 09 19:26:55.863834 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.863802 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:55.868088 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.868067 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7532-140609016882064"] Jul 09 19:26:55.923279 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.923257 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldj44\" (UniqueName: \"kubernetes.io/projected/a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d-kube-api-access-ldj44\") pod \"a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d\" (UID: \"a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d\") " Jul 09 19:26:55.925292 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.925236 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d-kube-api-access-ldj44" (OuterVolumeSpecName: "kube-api-access-ldj44") pod "a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" (UID: "a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d"). InnerVolumeSpecName "kube-api-access-ldj44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:26:55.996153 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.996126 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:55.996552 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.996539 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" containerName="test-curl-7532-140609016882064" Jul 09 19:26:55.996609 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.996554 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" containerName="test-curl-7532-140609016882064" Jul 09 19:26:55.996642 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.996623 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" containerName="test-curl-7532-140609016882064" Jul 09 19:26:55.999556 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:55.999542 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:56.004542 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.004521 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:56.024223 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.024200 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldj44\" (UniqueName: \"kubernetes.io/projected/a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d-kube-api-access-ldj44\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:26:56.066106 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.066071 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d" path="/var/lib/kubelet/pods/a3d5993d-fa7d-4c94-a98d-1ef3f2fcc98d/volumes" Jul 09 19:26:56.125305 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.125277 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsckf\" (UniqueName: \"kubernetes.io/projected/8633bc41-64d0-4939-aab2-62e144ed2789-kube-api-access-bsckf\") pod \"test-curl-7532-140609016883072\" (UID: \"8633bc41-64d0-4939-aab2-62e144ed2789\") " pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:56.226110 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.226035 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsckf\" (UniqueName: \"kubernetes.io/projected/8633bc41-64d0-4939-aab2-62e144ed2789-kube-api-access-bsckf\") pod \"test-curl-7532-140609016883072\" (UID: \"8633bc41-64d0-4939-aab2-62e144ed2789\") " pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:56.235293 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.235269 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsckf\" (UniqueName: \"kubernetes.io/projected/8633bc41-64d0-4939-aab2-62e144ed2789-kube-api-access-bsckf\") pod \"test-curl-7532-140609016883072\" (UID: \"8633bc41-64d0-4939-aab2-62e144ed2789\") " pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:56.309737 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.309703 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:56.443504 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.443477 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:56.445109 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:26:56.445084 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8633bc41_64d0_4939_aab2_62e144ed2789.slice/crio-e78cebfb753a286ba35e6e5082c9c6be9eced20f7dc4f49247a8b75188a3cd59 WatchSource:0}: Error finding container e78cebfb753a286ba35e6e5082c9c6be9eced20f7dc4f49247a8b75188a3cd59: Status 404 returned error can't find the container with id e78cebfb753a286ba35e6e5082c9c6be9eced20f7dc4f49247a8b75188a3cd59 Jul 09 19:26:56.679746 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.679720 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016882064" Jul 09 19:26:56.679916 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.679769 2587 scope.go:117] "RemoveContainer" containerID="ac72445363e8a9e2b5d3fc362f9a4cf080f85d8082221be00d0e20bd6a1054eb" Jul 09 19:26:56.681059 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:56.681029 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016883072" event={"ID":"8633bc41-64d0-4939-aab2-62e144ed2789","Type":"ContainerStarted","Data":"e78cebfb753a286ba35e6e5082c9c6be9eced20f7dc4f49247a8b75188a3cd59"} Jul 09 19:26:57.689774 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:57.689741 2587 generic.go:356] "Generic (PLEG): container finished" podID="8633bc41-64d0-4939-aab2-62e144ed2789" containerID="282f4039cb077b0a3ad223ab29afb0cd3c09d22776a517e6ee3dcbd374fe00d7" exitCode=0 Jul 09 19:26:57.690203 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:57.689832 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016883072" event={"ID":"8633bc41-64d0-4939-aab2-62e144ed2789","Type":"ContainerDied","Data":"282f4039cb077b0a3ad223ab29afb0cd3c09d22776a517e6ee3dcbd374fe00d7"} Jul 09 19:26:58.844545 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:58.844518 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:58.869134 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:58.869111 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7532-140609016883072_8633bc41-64d0-4939-aab2-62e144ed2789/test-curl-7532-140609016883072/0.log" Jul 09 19:26:58.898508 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:58.898478 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:58.902864 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:58.902841 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:58.952269 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:58.952210 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsckf\" (UniqueName: \"kubernetes.io/projected/8633bc41-64d0-4939-aab2-62e144ed2789-kube-api-access-bsckf\") pod \"8633bc41-64d0-4939-aab2-62e144ed2789\" (UID: \"8633bc41-64d0-4939-aab2-62e144ed2789\") " Jul 09 19:26:58.954214 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:58.954186 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8633bc41-64d0-4939-aab2-62e144ed2789-kube-api-access-bsckf" (OuterVolumeSpecName: "kube-api-access-bsckf") pod "8633bc41-64d0-4939-aab2-62e144ed2789" (UID: "8633bc41-64d0-4939-aab2-62e144ed2789"). InnerVolumeSpecName "kube-api-access-bsckf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:26:59.025959 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.025920 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:59.026338 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.026325 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8633bc41-64d0-4939-aab2-62e144ed2789" containerName="test-curl-7532-140609016883072" Jul 09 19:26:59.026390 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.026340 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="8633bc41-64d0-4939-aab2-62e144ed2789" containerName="test-curl-7532-140609016883072" Jul 09 19:26:59.026425 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.026408 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="8633bc41-64d0-4939-aab2-62e144ed2789" containerName="test-curl-7532-140609016883072" Jul 09 19:26:59.029293 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.029276 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:59.036252 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.036227 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:59.053832 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.053813 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsckf\" (UniqueName: \"kubernetes.io/projected/8633bc41-64d0-4939-aab2-62e144ed2789-kube-api-access-bsckf\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:26:59.155105 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.155067 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqtm4\" (UniqueName: \"kubernetes.io/projected/28690970-0282-4f90-b40d-6d7c91555cf0-kube-api-access-lqtm4\") pod \"test-curl-7532-140609016883072\" (UID: \"28690970-0282-4f90-b40d-6d7c91555cf0\") " pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:59.256114 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.256037 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqtm4\" (UniqueName: \"kubernetes.io/projected/28690970-0282-4f90-b40d-6d7c91555cf0-kube-api-access-lqtm4\") pod \"test-curl-7532-140609016883072\" (UID: \"28690970-0282-4f90-b40d-6d7c91555cf0\") " pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:59.266391 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.266364 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqtm4\" (UniqueName: \"kubernetes.io/projected/28690970-0282-4f90-b40d-6d7c91555cf0-kube-api-access-lqtm4\") pod \"test-curl-7532-140609016883072\" (UID: \"28690970-0282-4f90-b40d-6d7c91555cf0\") " pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:59.339062 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.339031 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:59.489077 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.489050 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:26:59.491103 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:26:59.491077 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28690970_0282_4f90_b40d_6d7c91555cf0.slice/crio-b43076f4525c59c0a4f31385c7c86f6820b65930f349a5046d475bc756a7a456 WatchSource:0}: Error finding container b43076f4525c59c0a4f31385c7c86f6820b65930f349a5046d475bc756a7a456: Status 404 returned error can't find the container with id b43076f4525c59c0a4f31385c7c86f6820b65930f349a5046d475bc756a7a456 Jul 09 19:26:59.702065 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.702039 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:26:59.702364 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.702041 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e78cebfb753a286ba35e6e5082c9c6be9eced20f7dc4f49247a8b75188a3cd59" Jul 09 19:26:59.703291 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.703262 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016883072" event={"ID":"28690970-0282-4f90-b40d-6d7c91555cf0","Type":"ContainerStarted","Data":"b43076f4525c59c0a4f31385c7c86f6820b65930f349a5046d475bc756a7a456"} Jul 09 19:26:59.721275 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:26:59.721241 2587 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7532-140609016883072" oldPodUID="8633bc41-64d0-4939-aab2-62e144ed2789" podUID="28690970-0282-4f90-b40d-6d7c91555cf0" Jul 09 19:27:00.065935 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:00.065900 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8633bc41-64d0-4939-aab2-62e144ed2789" path="/var/lib/kubelet/pods/8633bc41-64d0-4939-aab2-62e144ed2789/volumes" Jul 09 19:27:00.709455 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:00.709419 2587 generic.go:356] "Generic (PLEG): container finished" podID="28690970-0282-4f90-b40d-6d7c91555cf0" containerID="7106c97d8264544df9521e293d918dd106a69134b8d6a08c34a1fefaf4422f56" exitCode=0 Jul 09 19:27:00.709659 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:00.709502 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7532-140609016883072" event={"ID":"28690970-0282-4f90-b40d-6d7c91555cf0","Type":"ContainerDied","Data":"7106c97d8264544df9521e293d918dd106a69134b8d6a08c34a1fefaf4422f56"} Jul 09 19:27:01.859772 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:01.859747 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:27:01.886001 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:01.885953 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7532-140609016883072_28690970-0282-4f90-b40d-6d7c91555cf0/test-curl-7532-140609016883072/0.log" Jul 09 19:27:01.914165 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:01.914137 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:27:01.918592 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:01.918570 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7532-140609016883072"] Jul 09 19:27:01.980546 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:01.980483 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqtm4\" (UniqueName: \"kubernetes.io/projected/28690970-0282-4f90-b40d-6d7c91555cf0-kube-api-access-lqtm4\") pod \"28690970-0282-4f90-b40d-6d7c91555cf0\" (UID: \"28690970-0282-4f90-b40d-6d7c91555cf0\") " Jul 09 19:27:01.982478 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:01.982454 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28690970-0282-4f90-b40d-6d7c91555cf0-kube-api-access-lqtm4" (OuterVolumeSpecName: "kube-api-access-lqtm4") pod "28690970-0282-4f90-b40d-6d7c91555cf0" (UID: "28690970-0282-4f90-b40d-6d7c91555cf0"). InnerVolumeSpecName "kube-api-access-lqtm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:27:02.065781 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.065754 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28690970-0282-4f90-b40d-6d7c91555cf0" path="/var/lib/kubelet/pods/28690970-0282-4f90-b40d-6d7c91555cf0/volumes" Jul 09 19:27:02.081934 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.081910 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqtm4\" (UniqueName: \"kubernetes.io/projected/28690970-0282-4f90-b40d-6d7c91555cf0-kube-api-access-lqtm4\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:02.164905 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.164872 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:02.564166 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.564136 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:02.720849 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.720822 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7532-140609016883072" Jul 09 19:27:02.721059 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.720829 2587 scope.go:117] "RemoveContainer" containerID="7106c97d8264544df9521e293d918dd106a69134b8d6a08c34a1fefaf4422f56" Jul 09 19:27:02.736786 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.736760 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:02.837196 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:02.837100 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:03.021076 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:03.021040 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:09.613530 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:09.613426 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:09.712642 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:09.712604 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:09.774918 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:09.774891 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:09.967705 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:09.967671 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:10.216516 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:10.216484 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:10.793881 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:10.793850 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:10.893837 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:10.893802 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:10.995174 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:10.995135 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:11.170963 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:11.170911 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:17.827048 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:17.827008 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:17.897364 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:17.897326 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:18.083736 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:18.083656 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s"] Jul 09 19:27:18.083892 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:18.083873 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" containerID="cri-o://7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715" gracePeriod=30 Jul 09 19:27:18.090838 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:18.090812 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:18.323630 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:18.323600 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:23.351183 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.351128 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:27:23.377582 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377557 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4522dc-7af5-491f-bcee-f8af65b704c6-istiod-ca-cert\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377731 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377600 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-socket\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377731 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377621 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht67m\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-kube-api-access-ht67m\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377731 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377658 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-certs\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377731 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377704 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-podinfo\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377956 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377738 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-credential-socket\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377956 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377801 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-token\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377956 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377831 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-envoy\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.377956 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377865 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-data\") pod \"ba4522dc-7af5-491f-bcee-f8af65b704c6\" (UID: \"ba4522dc-7af5-491f-bcee-f8af65b704c6\") " Jul 09 19:27:23.378161 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.377968 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba4522dc-7af5-491f-bcee-f8af65b704c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 19:27:23.378161 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.378085 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:27:23.378286 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.378252 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:27:23.378379 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.378358 2587 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-credential-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.378438 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.378387 2587 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4522dc-7af5-491f-bcee-f8af65b704c6-istiod-ca-cert\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.378438 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.378401 2587 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-socket\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.378542 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.378461 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:27:23.379072 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.379043 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:27:23.380381 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.380345 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-kube-api-access-ht67m" (OuterVolumeSpecName: "kube-api-access-ht67m") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "kube-api-access-ht67m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:27:23.380479 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.380417 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 19:27:23.380479 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.380451 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 19:27:23.380479 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.380464 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba4522dc-7af5-491f-bcee-f8af65b704c6" (UID: "ba4522dc-7af5-491f-bcee-f8af65b704c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 19:27:23.479502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.479477 2587 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-workload-certs\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.479502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.479497 2587 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-podinfo\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.479502 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.479506 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-token\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.479676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.479515 2587 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-envoy\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.479676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.479523 2587 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4522dc-7af5-491f-bcee-f8af65b704c6-istio-data\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.479676 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.479531 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht67m\" (UniqueName: \"kubernetes.io/projected/ba4522dc-7af5-491f-bcee-f8af65b704c6-kube-api-access-ht67m\") on node \"ip-10-0-134-249.ec2.internal\" DevicePath \"\"" Jul 09 19:27:23.680780 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.680755 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zbtln_f3b80d39-2997-47cb-ba8b-84c98e0e9100/manager/0.log" Jul 09 19:27:23.803649 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.803626 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dxk7l_14deb11e-0a77-4a7a-82be-50f03c77564b/manager/0.log" Jul 09 19:27:23.840729 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.840700 2587 generic.go:356] "Generic (PLEG): container finished" podID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerID="7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715" exitCode=0 Jul 09 19:27:23.840901 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.840757 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" Jul 09 19:27:23.840901 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.840792 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" event={"ID":"ba4522dc-7af5-491f-bcee-f8af65b704c6","Type":"ContainerDied","Data":"7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715"} Jul 09 19:27:23.840901 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.840884 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s" event={"ID":"ba4522dc-7af5-491f-bcee-f8af65b704c6","Type":"ContainerDied","Data":"6eb4d668acf873f2458806e72b0a019b805eb3912b2a6736458ece55785202af"} Jul 09 19:27:23.841091 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.840914 2587 scope.go:117] "RemoveContainer" containerID="7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715" Jul 09 19:27:23.850234 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.850218 2587 scope.go:117] "RemoveContainer" containerID="9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32" Jul 09 19:27:23.857982 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.857966 2587 scope.go:117] "RemoveContainer" containerID="7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715" Jul 09 19:27:23.858213 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:27:23.858197 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715\": container with ID starting with 7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715 not found: ID does not exist" containerID="7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715" Jul 09 19:27:23.858263 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.858221 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715"} err="failed to get container status \"7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715\": rpc error: code = NotFound desc = could not find container \"7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715\": container with ID starting with 7794aa7877e5e686ede47bb8fd12d7e0d28703e4f79e14e1689d3d3419fd6715 not found: ID does not exist" Jul 09 19:27:23.858263 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.858237 2587 scope.go:117] "RemoveContainer" containerID="9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32" Jul 09 19:27:23.858488 ip-10-0-134-249 kubenswrapper[2587]: E0709 19:27:23.858473 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32\": container with ID starting with 9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32 not found: ID does not exist" containerID="9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32" Jul 09 19:27:23.858528 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.858491 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32"} err="failed to get container status \"9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32\": rpc error: code = NotFound desc = could not find container \"9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32\": container with ID starting with 9181fad96be8ca644124903d373f7cb926100e8ade51134efdaa05e27d844e32 not found: ID does not exist" Jul 09 19:27:23.867055 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.867034 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s"] Jul 09 19:27:23.872102 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.872082 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a966ef-openshift-default-7548c9c787-l9v7s"] Jul 09 19:27:23.929300 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:23.929281 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b7f6c8fdb-rb9hw_cfe97171-2f03-4a94-b8ab-aa32e3d8681d/manager/0.log" Jul 09 19:27:24.046089 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:24.046012 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rzjw2_48a521dd-49a5-4fd1-b6ea-624e4588db80/server/0.log" Jul 09 19:27:24.068810 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:24.068779 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" path="/var/lib/kubelet/pods/ba4522dc-7af5-491f-bcee-f8af65b704c6/volumes" Jul 09 19:27:24.164648 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:24.164627 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c6rzh_78afe730-cd3d-4ada-89a4-b2b54bdcbbda/manager/0.log" Jul 09 19:27:24.560803 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:24.560780 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt_75e55877-5527-48d2-8709-729cc1f01eaf/copy-manifests/0.log" Jul 09 19:27:24.570828 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:24.570811 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt_75e55877-5527-48d2-8709-729cc1f01eaf/manager/0.log" Jul 09 19:27:25.024707 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:25.024680 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wv45g_655e7b9b-669d-4ef8-bfbe-da9dbe696137/postgres/0.log" Jul 09 19:27:26.484733 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:26.484710 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7774t_196641c8-2d05-4612-8365-30d9de32d76a/manager/0.log" Jul 09 19:27:26.605567 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:26.605536 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wdvrh_ac456cf2-4b6f-4dd2-9aa9-16f34109a37b/manager/0.log" Jul 09 19:27:26.836904 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:26.836828 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-24ptc_875f2617-eb22-49cc-a8e8-049982697a03/registry-server/0.log" Jul 09 19:27:26.959357 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:26.959329 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6rk7g_975e2533-0ce3-4706-922f-98f13fa74100/manager/0.log" Jul 09 19:27:27.904289 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:27.904259 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86888fb94c-9vwhx_74c082ca-4fef-4f6e-8ef9-a474f53789fa/kube-auth-proxy/0.log" Jul 09 19:27:28.030296 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:28.030265 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hf5ch_e0225e57-c471-4ef4-ae65-f623a539d966/istio-proxy/1.log" Jul 09 19:27:28.152653 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:28.152627 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4gsnx_bce1ad29-3d4a-479d-844b-a1b22d6ac880/payload-pre-processing/0.log" Jul 09 19:27:28.271524 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:28.271451 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9lz4z_bf4b8280-2454-4da9-9cc5-a9a15587faf4/payload-processing/0.log" Jul 09 19:27:28.742199 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:28.742163 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-w7522_e5b7ca73-8b96-43c4-9995-cfd454087b6c/main/0.log" Jul 09 19:27:28.866235 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:28.866206 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wkgp4_c17e81ba-4053-4f5d-afb8-fc0200d2c5ce/main/0.log" Jul 09 19:27:28.988573 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:28.988546 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-6m25q_999cefec-d3db-47b2-9e0f-05cce59d30d5/main/0.log" Jul 09 19:27:33.685255 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685223 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qgfk6/must-gather-hdkcn"] Jul 09 19:27:33.685708 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685609 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" Jul 09 19:27:33.685708 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685619 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" Jul 09 19:27:33.685708 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685639 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28690970-0282-4f90-b40d-6d7c91555cf0" containerName="test-curl-7532-140609016883072" Jul 09 19:27:33.685708 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685645 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="28690970-0282-4f90-b40d-6d7c91555cf0" containerName="test-curl-7532-140609016883072" Jul 09 19:27:33.685708 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685710 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" Jul 09 19:27:33.685873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685719 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" Jul 09 19:27:33.685873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685724 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="28690970-0282-4f90-b40d-6d7c91555cf0" containerName="test-curl-7532-140609016883072" Jul 09 19:27:33.685873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685793 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" Jul 09 19:27:33.685873 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.685798 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4522dc-7af5-491f-bcee-f8af65b704c6" containerName="istio-proxy" Jul 09 19:27:33.690685 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.690667 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:33.691487 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.691458 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:33.693446 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.693424 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qgfk6\"/\"openshift-service-ca.crt\"" Jul 09 19:27:33.694447 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.694427 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qgfk6\"/\"default-dockercfg-2s96h\"" Jul 09 19:27:33.694553 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.694434 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qgfk6\"/\"kube-root-ca.crt\"" Jul 09 19:27:33.698971 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.698934 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qgfk6/must-gather-hdkcn"] Jul 09 19:27:33.762936 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.762908 2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" limit=1024 Jul 09 19:27:33.774894 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.774871 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf67q\" (UniqueName: \"kubernetes.io/projected/10608d46-13ea-42ee-8a0f-9e47cb2928fe-kube-api-access-cf67q\") pod \"must-gather-hdkcn\" (UID: \"10608d46-13ea-42ee-8a0f-9e47cb2928fe\") " pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:33.774991 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.774915 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10608d46-13ea-42ee-8a0f-9e47cb2928fe-must-gather-output\") pod \"must-gather-hdkcn\" (UID: \"10608d46-13ea-42ee-8a0f-9e47cb2928fe\") " pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:33.876019 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.875986 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf67q\" (UniqueName: \"kubernetes.io/projected/10608d46-13ea-42ee-8a0f-9e47cb2928fe-kube-api-access-cf67q\") pod \"must-gather-hdkcn\" (UID: \"10608d46-13ea-42ee-8a0f-9e47cb2928fe\") " pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:33.876196 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.876059 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10608d46-13ea-42ee-8a0f-9e47cb2928fe-must-gather-output\") pod \"must-gather-hdkcn\" (UID: \"10608d46-13ea-42ee-8a0f-9e47cb2928fe\") " pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:33.876410 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.876391 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10608d46-13ea-42ee-8a0f-9e47cb2928fe-must-gather-output\") pod \"must-gather-hdkcn\" (UID: \"10608d46-13ea-42ee-8a0f-9e47cb2928fe\") " pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:33.886238 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:33.886191 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf67q\" (UniqueName: \"kubernetes.io/projected/10608d46-13ea-42ee-8a0f-9e47cb2928fe-kube-api-access-cf67q\") pod \"must-gather-hdkcn\" (UID: \"10608d46-13ea-42ee-8a0f-9e47cb2928fe\") " pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:34.001751 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:34.001663 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qgfk6/must-gather-hdkcn" Jul 09 19:27:34.138322 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:34.138296 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qgfk6/must-gather-hdkcn"] Jul 09 19:27:34.140078 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:27:34.140046 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10608d46_13ea_42ee_8a0f_9e47cb2928fe.slice/crio-4ef6f0df3cb153c2f2db2ae9fc1fef48ae42ce87d72c5641cee7e766d07cb400 WatchSource:0}: Error finding container 4ef6f0df3cb153c2f2db2ae9fc1fef48ae42ce87d72c5641cee7e766d07cb400: Status 404 returned error can't find the container with id 4ef6f0df3cb153c2f2db2ae9fc1fef48ae42ce87d72c5641cee7e766d07cb400 Jul 09 19:27:34.898262 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:34.898221 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qgfk6/must-gather-hdkcn" event={"ID":"10608d46-13ea-42ee-8a0f-9e47cb2928fe","Type":"ContainerStarted","Data":"4ef6f0df3cb153c2f2db2ae9fc1fef48ae42ce87d72c5641cee7e766d07cb400"} Jul 09 19:27:35.905578 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:35.905542 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qgfk6/must-gather-hdkcn" event={"ID":"10608d46-13ea-42ee-8a0f-9e47cb2928fe","Type":"ContainerStarted","Data":"d2e9bcdc387d0de16f49da502e5be06733c08754f891a44807f145924df064ac"} Jul 09 19:27:35.906046 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:35.905588 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qgfk6/must-gather-hdkcn" event={"ID":"10608d46-13ea-42ee-8a0f-9e47cb2928fe","Type":"ContainerStarted","Data":"106d32adc804bfc834c586dcf662bf7a56132917ebd0a7eb1b66bb530bc33ce4"} Jul 09 19:27:35.927340 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:35.927291 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qgfk6/must-gather-hdkcn" podStartSLOduration=2.035942157 podStartE2EDuration="2.927275955s" totalImagesPullingTime="891.333798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:27:35.926714432 +0000 UTC m=+2542.427673473" watchObservedRunningTime="2026-07-09 19:27:35.927275955 +0000 UTC m=+2542.428234993" Jul 09 19:27:36.819248 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:36.819221 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vf5zs_88545bfb-fc18-4f99-ab53-894af89974a2/global-pull-secret-syncer/0.log" Jul 09 19:27:36.877572 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:36.877545 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bg9n2_dc77411f-12b3-40e2-b71a-b338eabd5065/konnectivity-agent/0.log" Jul 09 19:27:36.967163 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:36.967133 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-249.ec2.internal_bfa3c3be93b6e0164ff4342e8421b411/haproxy/0.log" Jul 09 19:27:41.790733 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:41.790703 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7774t_196641c8-2d05-4612-8365-30d9de32d76a/manager/0.log" Jul 09 19:27:41.822520 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:41.822494 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wdvrh_ac456cf2-4b6f-4dd2-9aa9-16f34109a37b/manager/0.log" Jul 09 19:27:41.910374 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:41.910334 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-24ptc_875f2617-eb22-49cc-a8e8-049982697a03/registry-server/0.log" Jul 09 19:27:42.072013 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:42.067905 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6rk7g_975e2533-0ce3-4706-922f-98f13fa74100/manager/0.log" Jul 09 19:27:44.074906 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:44.071624 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2sxcb_fa2bf4cf-c6bc-4981-8a80-5f148b082519/node-exporter/0.log" Jul 09 19:27:44.108748 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:44.108718 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2sxcb_fa2bf4cf-c6bc-4981-8a80-5f148b082519/kube-rbac-proxy/0.log" Jul 09 19:27:44.139817 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:44.139737 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2sxcb_fa2bf4cf-c6bc-4981-8a80-5f148b082519/init-textfile/0.log" Jul 09 19:27:46.538697 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.538660 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn"] Jul 09 19:27:46.543426 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.543391 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.555783 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.555621 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn"] Jul 09 19:27:46.720399 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.720367 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-sys\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.721251 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.721227 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-lib-modules\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.721497 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.721462 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-podres\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.721750 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.721732 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xghb\" (UniqueName: \"kubernetes.io/projected/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-kube-api-access-9xghb\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.721896 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.721881 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-proc\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823384 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823290 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-sys\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823384 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823340 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-lib-modules\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823384 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823383 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-podres\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823688 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823442 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-sys\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823688 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823481 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xghb\" (UniqueName: \"kubernetes.io/projected/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-kube-api-access-9xghb\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823688 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823510 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-proc\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823688 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823588 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-lib-modules\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823688 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823633 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-proc\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.823688 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.823670 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-podres\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.834413 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.834376 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xghb\" (UniqueName: \"kubernetes.io/projected/dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce-kube-api-access-9xghb\") pod \"perf-node-gather-daemonset-c8qzn\" (UID: \"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce\") " pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.858346 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.858306 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:46.882506 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.882403 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/2.log" Jul 09 19:27:46.887830 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:46.887550 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8g5bp_de304309-203f-46ba-ba4d-024e0ab59b8c/console-operator/3.log" Jul 09 19:27:47.059113 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:47.057797 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn"] Jul 09 19:27:47.064797 ip-10-0-134-249 kubenswrapper[2587]: W0709 19:27:47.064602 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddd9d3df0_8d0b_44e7_b9a9_50090d3bfbce.slice/crio-23154c5de5013843489f9f86e5a01ddcafcd743d394167c85089e9eb23cad215 WatchSource:0}: Error finding container 23154c5de5013843489f9f86e5a01ddcafcd743d394167c85089e9eb23cad215: Status 404 returned error can't find the container with id 23154c5de5013843489f9f86e5a01ddcafcd743d394167c85089e9eb23cad215 Jul 09 19:27:48.012401 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:48.012359 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" event={"ID":"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce","Type":"ContainerStarted","Data":"5a44e7643d4d00480661d77cbbf4e3be20fedc65434569d97a008f51c85b5e48"} Jul 09 19:27:48.012868 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:48.012463 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" event={"ID":"dd9d3df0-8d0b-44e7-b9a9-50090d3bfbce","Type":"ContainerStarted","Data":"23154c5de5013843489f9f86e5a01ddcafcd743d394167c85089e9eb23cad215"} Jul 09 19:27:48.013350 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:48.013298 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:48.033688 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:48.033634 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" podStartSLOduration=2.033614203 podStartE2EDuration="2.033614203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:27:48.031321593 +0000 UTC m=+2554.532280633" watchObservedRunningTime="2026-07-09 19:27:48.033614203 +0000 UTC m=+2554.534573241" Jul 09 19:27:48.101707 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:48.101679 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-4r8w9_7ae20ec9-71d1-44c8-a486-040683dfa34f/volume-data-source-validator/0.log" Jul 09 19:27:49.148142 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:49.148117 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9p7tm_949afb45-35a9-4750-99a8-9bf32d8f8e25/dns/0.log" Jul 09 19:27:49.172934 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:49.172902 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9p7tm_949afb45-35a9-4750-99a8-9bf32d8f8e25/kube-rbac-proxy/0.log" Jul 09 19:27:49.391252 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:49.391223 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r2x85_7fb5d87d-2207-4215-900e-9f5cbde935de/dns-node-resolver/0.log" Jul 09 19:27:49.972108 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:49.972077 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6745f944d7-jmsjg_c8627e74-def1-4cd6-8561-440a4f05b355/registry/0.log" Jul 09 19:27:49.997085 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:49.997053 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dh7hx_542ce291-5b21-4361-b4a1-58e67e29c530/node-ca/0.log" Jul 09 19:27:51.190426 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:51.190399 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86888fb94c-9vwhx_74c082ca-4fef-4f6e-8ef9-a474f53789fa/kube-auth-proxy/0.log" Jul 09 19:27:51.223460 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:51.223431 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hf5ch_e0225e57-c471-4ef4-ae65-f623a539d966/istio-proxy/0.log" Jul 09 19:27:51.261061 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:51.261028 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hf5ch_e0225e57-c471-4ef4-ae65-f623a539d966/istio-proxy/1.log" Jul 09 19:27:51.305924 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:51.305892 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4gsnx_bce1ad29-3d4a-479d-844b-a1b22d6ac880/payload-pre-processing/0.log" Jul 09 19:27:51.362328 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:51.362291 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9lz4z_bf4b8280-2454-4da9-9cc5-a9a15587faf4/payload-processing/0.log" Jul 09 19:27:52.150356 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:52.150313 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wlxp4_6fcf474d-2db5-4a19-803c-414e4ab64a2b/serve-healthcheck-canary/0.log" Jul 09 19:27:52.648239 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:52.648209 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-5r4rb_5400ba01-ed2f-40e4-8c04-85be98f44d1a/insights-operator/0.log" Jul 09 19:27:52.649655 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:52.649632 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-5r4rb_5400ba01-ed2f-40e4-8c04-85be98f44d1a/insights-operator/1.log" Jul 09 19:27:52.747409 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:52.747377 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6wkrq_026a8fa5-9e41-4153-98f4-439bb0f833a9/kube-rbac-proxy/0.log" Jul 09 19:27:52.771866 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:52.771844 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6wkrq_026a8fa5-9e41-4153-98f4-439bb0f833a9/exporter/0.log" Jul 09 19:27:52.796627 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:52.796601 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6wkrq_026a8fa5-9e41-4153-98f4-439bb0f833a9/extractor/0.log" Jul 09 19:27:55.009406 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.009365 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zbtln_f3b80d39-2997-47cb-ba8b-84c98e0e9100/manager/0.log" Jul 09 19:27:55.039841 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.039815 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qgfk6/perf-node-gather-daemonset-c8qzn" Jul 09 19:27:55.180700 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.180665 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dxk7l_14deb11e-0a77-4a7a-82be-50f03c77564b/manager/0.log" Jul 09 19:27:55.307462 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.307383 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b7f6c8fdb-rb9hw_cfe97171-2f03-4a94-b8ab-aa32e3d8681d/manager/0.log" Jul 09 19:27:55.331841 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.331806 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rzjw2_48a521dd-49a5-4fd1-b6ea-624e4588db80/server/0.log" Jul 09 19:27:55.416624 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.416586 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c6rzh_78afe730-cd3d-4ada-89a4-b2b54bdcbbda/manager/0.log" Jul 09 19:27:55.587983 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.587898 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt_75e55877-5527-48d2-8709-729cc1f01eaf/manager/0.log" Jul 09 19:27:55.616030 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:55.616000 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c9d9c8f47-c9lwt_75e55877-5527-48d2-8709-729cc1f01eaf/copy-manifests/0.log" Jul 09 19:27:56.618669 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:56.618642 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86888fb94c-9vwhx_74c082ca-4fef-4f6e-8ef9-a474f53789fa/kube-auth-proxy/0.log" Jul 09 19:27:56.645175 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:56.645137 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hf5ch_e0225e57-c471-4ef4-ae65-f623a539d966/istio-proxy/0.log" Jul 09 19:27:56.682312 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:56.682282 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hf5ch_e0225e57-c471-4ef4-ae65-f623a539d966/istio-proxy/1.log" Jul 09 19:27:56.722122 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:56.722092 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4gsnx_bce1ad29-3d4a-479d-844b-a1b22d6ac880/payload-pre-processing/0.log" Jul 09 19:27:56.770928 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:56.770900 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9lz4z_bf4b8280-2454-4da9-9cc5-a9a15587faf4/payload-processing/0.log" Jul 09 19:27:58.254192 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:58.254149 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86888fb94c-9vwhx_74c082ca-4fef-4f6e-8ef9-a474f53789fa/kube-auth-proxy/0.log" Jul 09 19:27:58.286480 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:58.286442 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hf5ch_e0225e57-c471-4ef4-ae65-f623a539d966/istio-proxy/0.log" Jul 09 19:27:58.327628 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:58.327589 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hf5ch_e0225e57-c471-4ef4-ae65-f623a539d966/istio-proxy/1.log" Jul 09 19:27:58.366252 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:58.366190 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-4gsnx_bce1ad29-3d4a-479d-844b-a1b22d6ac880/payload-pre-processing/0.log" Jul 09 19:27:58.413813 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:27:58.413774 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-9lz4z_bf4b8280-2454-4da9-9cc5-a9a15587faf4/payload-processing/0.log" Jul 09 19:28:02.310059 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:02.310032 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-zptbt_d551c345-9a33-408c-b17e-e394101f94be/migrator/0.log" Jul 09 19:28:02.335432 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:02.335401 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-zptbt_d551c345-9a33-408c-b17e-e394101f94be/graceful-termination/0.log" Jul 09 19:28:02.724696 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:02.724662 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-lj7ph_84b890da-cc0c-4274-9b59-7839dad6a083/kube-storage-version-migrator-operator/1.log" Jul 09 19:28:02.725808 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:02.725787 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-lj7ph_84b890da-cc0c-4274-9b59-7839dad6a083/kube-storage-version-migrator-operator/0.log" Jul 09 19:28:03.916919 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:03.916892 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4rwj_adfe3fbd-54e0-4772-8240-4afdd653ac76/kube-multus-additional-cni-plugins/0.log" Jul 09 19:28:03.941217 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:03.941189 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4rwj_adfe3fbd-54e0-4772-8240-4afdd653ac76/egress-router-binary-copy/0.log" Jul 09 19:28:03.963340 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:03.963319 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4rwj_adfe3fbd-54e0-4772-8240-4afdd653ac76/cni-plugins/0.log" Jul 09 19:28:03.986553 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:03.986532 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4rwj_adfe3fbd-54e0-4772-8240-4afdd653ac76/bond-cni-plugin/0.log" Jul 09 19:28:04.009814 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:04.009794 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4rwj_adfe3fbd-54e0-4772-8240-4afdd653ac76/routeoverride-cni/0.log" Jul 09 19:28:04.041577 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:04.041553 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4rwj_adfe3fbd-54e0-4772-8240-4afdd653ac76/whereabouts-cni-bincopy/0.log" Jul 09 19:28:04.066630 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:04.066601 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4rwj_adfe3fbd-54e0-4772-8240-4afdd653ac76/whereabouts-cni/0.log" Jul 09 19:28:04.459202 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:04.459174 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fvbjs_651837c0-e0d3-4abe-94aa-d62e0d8cde1e/kube-multus/0.log" Jul 09 19:28:04.541380 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:04.541347 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b62dq_f30cc372-130d-417c-a1c4-2754e9d9f80d/network-metrics-daemon/0.log" Jul 09 19:28:04.563136 ip-10-0-134-249 kubenswrapper[2587]: I0709 19:28:04.563111 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b62dq_f30cc372-130d-417c-a1c4-2754e9d9f80d/kube-rbac-proxy/0.log"